Using neural networks to perform fault detection in autonomous driving applications

ABSTRACT

In various examples, motifs, watermarks, and/or signature inputs are applied to a deep neural network (DNN) to detect faults in underlying hardware and/or software executing the DNN. Information corresponding to the motifs, watermarks, and/or signatures may be compared to the outputs of the DNN generated using the motifs, watermarks and/or signatures. When a the accuracy of the predictions are below a threshold, or do not correspond to the expected predictions of the DNN, the hardware and/or software may be determined to have a fault—such as a transient, an intermittent, or a permanent fault. Where a fault is determined, portions of the system that rely on the computations of the DNN may be shut down, or redundant systems may be used in place of the primary system. Where no fault is determined, the computations of the DNN may be relied upon by the system.

BACKGROUND

Autonomous driving systems and advanced driver assistance systems (ADAS)may leverage sensors (such as cameras) to perform various tasks—such aswithout limitation, lane keeping, lane changing, lane assignment, lanedetection, object detection, path planning, camera calibration, andlocalization. To perform many of these tasks, machine learningmodels—and specifically deep neural networks (DNNs)—may be used toperform at least some of the processing. As a result, for these systemsto operate with a level of safety required for autonomous orsemi-autonomous driving functionality, the machine learning models needto execute as intended over the life of their implementation.

However, software and/or hardware used to execute these DNNs may becompromised by a variety of sources—resulting in transient faults and/orpermanent faults—that may lead to inaccurate predictions that maypotentially compromise the effectiveness of the DNNs. As examples,potential causes of faults in DNNs may include hardware faults inprocessing units executing the DNNs, and/or software faults of theunderlying DNN. As such, the ability to robustly and accurately detectfaults associated with the DNNs may allow the autonomous and/or ADASsystems to make critical decisions in real-time or near real time—suchas to suggest or implement corrective measures for effective and safedriving. For example, accurately, efficiently, and timely identifyingDNN faults may allow a system to identify when operations of the systemthat rely on predictions of the DNN are impaired and, as a result, allowthe system to perform corrective operations, such as handing controlback to a human driver or executing a safety maneuver (e.g., pulling tothe side of the road).

In conventional systems, resilience and/or fault coverage of a DNN maybe determined by redundant execution of the DNN. For example, two ormore instances of a DNN may be executed on the same input data, and theoutputs of the two or more instances may be compared to one another todetermine if any discrepancies exist. However, running multipleinstances of a DNN is both memory and processing intensive, alsorequiring—in some scenarios—additional hardware utilization for runningthe concurrent instances. In addition, these conventional systemsrequire that the outputs of the two or more instances of the DNN becompared, often at each iteration, further adding to the computationalexpense. Due to the computational burden of these conventionalapproaches on the underlying system, executing these processes mayprevent real-time or near real-time fault detection capabilities.

SUMMARY

Embodiments of the present disclosure relate to fault detection inneural networks. Systems and methods are disclosed that use techniquesto detect transient and/or permanent faults in one or more components ofa hardware and/or software system using a single instance of a neuralnetwork.

In contrast to conventional systems, such as those described above, thesystem of the present disclosure may implement a deep neural network(DNN) to detect faults in at least the hardware and/or softwareexecuting the DNN by using motifs, watermarks, and/or signatures (e.g.,signature images) as inputs- or modifications to inputs—of the DNN. As aresult of the processes described herein, fault analysis with increasedfault coverage as compared to conventional approaches may be performedusing a single instance of a DNN—thereby reducing the computationalexpense and enabling deployment in real-time or near real-time. Forexample, due to the real-time capability of the present system, faultdetection in DNNs may be executed as part of a built-in self-test (BIST)system operating in deployment.

Motifs, watermarks, and/or signatures may be used as input, or appendedto inputs (e.g., appended to sensor data), for the DNN. In non-limitingembodiments, the motifs, watermarks, and/or signatures may be selectedto leverage the downstream task that the DNN is already trained for. Forexample, where the DNN is trained for object detection, or specificallydetection of vehicles, pedestrians, and bicyclists, the motifs,watermarks, and/or signatures may represent vehicles, pedestrians,and/or bicyclists. Predictions of the DNN with respect to the motifs,watermarks, and/or signatures may be compared to expected predictionswith respect to the same, and inconsistencies may be analyzed todetermine whether a fault exists. By appending and/or using the motifs,watermarks, and/or signatures as input, the motifs, watermarks, and/orsignatures may be processed by the single instance of a DNN—therebyreducing compute resources as compared to conventional systems thatrequire multiple instances of a DNN for fault detection whilesimultaneously decreasing run-time to allow for real-time deployment.

BRIEF DESCRIPTION OF THE DRAWINGS

The present systems and methods for fault detection in neural networksare described in detail below with reference to the attached drawingfigures, wherein:

FIG. 1 is an example data flow diagram illustrating a process fordetecting transient faults in neural networks, in accordance with someembodiments of the present disclosure;

FIG. 2 includes an example illustration of a neural network trained todetect objects, in accordance with some embodiments of the presentdisclosure;

FIGS. 3A-3B include example illustrations of appending motifs to aninput image, in accordance with some embodiments of the presentdisclosure;

FIG. 4 is an example illustration of a process for analyzing outputs ofa network for fault detection, in accordance with some embodiments ofthe present disclosure;

FIG. 5 is an example illustration of a process for detecting faults in aneural network, in accordance with some embodiments of the presentdisclosure;

FIG. 6 is a flow diagram showing a method for detecting faults in aneural network using motifs and/or watermarks, in accordance with someembodiments of the present disclosure;

FIG. 7 is an example data flow diagram illustrating a process fordetecting faults in a neural network, in accordance with someembodiments of the present disclosure;

FIGS. 8A-8C include example illustrations of signature inputs used todetermine faults in neural networks, in accordance with some embodimentsof the present disclosure;

FIG. 9 is a flow diagram showing a method for detecting faults in aneural network using signatures, in accordance with some embodiments ofthe present disclosure;

FIG. 10A is an illustration of an example autonomous vehicle, inaccordance with some embodiments of the present disclosure;

FIG. 10B is an example of camera locations and fields of view for theexample autonomous vehicle of FIG. 10A, in accordance with someembodiments of the present disclosure;

FIG. 10C is a block diagram of an example system architecture for theexample autonomous vehicle of FIG. 10A, in accordance with someembodiments of the present disclosure;

FIG. 10D is a system diagram for communication between cloud-basedserver(s) and the example autonomous vehicle of FIG. 10A, in accordancewith some embodiments of the present disclosure; and

FIG. 11 is a block diagram of an example computing device suitable foruse in implementing some embodiments of the present disclosure.

DETAILED DESCRIPTION

Systems and methods are disclosed related to fault detection in neuralnetworks. The systems and methods described herein may be used inaugmented reality, virtual reality, robotics, security and surveillance,medical imaging, autonomous or semi-autonomous machine applications,gaming, financial services, marketing and sales, governmentapplications, oil and gas technology, weather forecasting, and/or anyother technology spaces where neural networks or other machine learningmodels are used—and thus may benefit from fault detection with respectto the same. Although the present disclosure may be described withrespect to an example autonomous vehicle 1000 (alternatively referred toherein as “vehicle 1000” or “autonomous vehicle 1000,” an example ofwhich is described with respect to FIGS. 10A-10D), this is not intendedto be limiting. For example, the systems and methods described hereinmay be used by, without limitation, non-autonomous vehicles,semi-autonomous vehicles (e.g., in one or more adaptive driverassistance systems (ADAS)), robots, warehouse vehicles, off-roadvehicles, flying vessels, boats, shuttles, emergency response vehicles,motorcycles, electric or motorized bicycles, aircraft, constructionvehicles, underwater craft, drones, and/or other vehicle types. Inaddition, although the faults of the present disclosure may referprimarily to any transient or permanent faults related to hardwareand/or software of the underlying system, this is not intended to belimiting. For example, where transient fault detection is describedherein, such examples may also be used for permanent fault detection,and vice versa. Similarly, in addition to or alternatively fromtransient and permanent faults, the systems and methods of the presentdisclosure may be leveraged for intermittent fault detection and/orother types of fault detection, without departing from the scope of thepresent disclosure.

Transient Fault Detection Method

As described herein, in contrast to conventional approaches of neuralnetwork fault detection, the current system detects transient and/orpermanent faults in DNNs using motifs, watermarks, and/or signatures(e.g., signature images) as input—or as modifications to inputs—of theDNN. As a result, fault analysis of the DNN may be performed using asingle instance of a DNN—thereby reducing the computational expense ascompared to conventional systems and enabling deployment in real-time ornear real-time. In some non-limiting embodiments, such as for transientfault detection, motifs or watermarks may be appended to input data(e.g., image data representative of images) to enable fault detectionusing the DNNs. For example, motifs or watermarks corresponding to thetypes of predictions the DNN is trained to make may be appended to theinput data, thereby leveraging the already trained DNN to make faultpredictions. In such embodiments, the predictions of the DNN withrespect to the motifs or watermarks may be compared to expectedpredictions of the DNN with respect to the motifs or watermarks, andinconsistencies may be analyzed to determine whether a fault is present.

With reference to detecting a fault(s) using a DNN, sensor data (e.g.,images, videos, depth maps, point clouds, etc.) may be received fromsensors (e.g., cameras, LIDAR, RADAR, etc.) disposed on or otherwiseassociated with a vehicle (e.g., an autonomous vehicle, asemi-autonomous vehicle, etc.). Motifs may be introduced (e.g.,appended) to the sensor data to generate input data representing thesensor data and the motifs. The input data may be applied to a DNN thatis trained to generate predictions (e.g., locations and/or classes ofobjects, locations of lines, lanes, signs, etc., intersections, paths,etc.) based on the input data. As such, the motifs or watermarks maycorrespond to the types and/or classes of objects, lines, etc. that theDNN is trained to predict.

The DNN may then output predictions corresponding to the sensor data andthe motifs included in the input data. A fault detector may then be usedto compare the predictions of the DNN to expected predictionscorresponding to the motifs. For example, the expected predictions mayinclude actual locations and/or class labels of the motifs, which mayhave been stored at the time the motifs are introduced into the inputdata. The comparison may include comparing locations and/or class labelscorresponding to the input data to ensure that the predictions at leastinclude accurate or expected predictions with respect to the motifs.Based on the comparison, the accuracy or resiliency of the network maybe determined, such as by identifying if the predictions of the DNN donot correspond to the motif information known to the fault detector, orthe predictions are outside of a threshold accuracy with respect to themotif information.

For example, when the expected predictions corresponding to the motifsare represented in the predictions of the DNN, the DNN predictions maybe determined to be accurate, and the DNN, or the supporting hardwareand/or software, may be considered fault free at least with respect tothe processing of the DNN. Similarly, when the expected predictionscorresponding to the motifs are not represented in the predictions ofthe DNN, the DNN predictions may be determined to be inaccurate, thepredictions may be thrown out or disregarded by the system, and/or theDNN, or the supporting hardware and/or software may be considered toinclude a fault—such as a transient fault. In such examples, where afault is detected, corrective measures may be taken, such as to handover the control of the vehicle to the driver, perform a safetymaneuver, and/or to offload or transfer processing of the DNN to othercomponents—e.g., a redundant architecture. In some examples, the motifsmay be applied at each iteration of sensor data input to the DNN. Inthis way, the system may account for faults on an ongoing basis, therebyincreasing safety of the system. However, this is not intended to belimiting, and the motifs may be appended at a different interval, suchas randomly, every other instance of the input data, every thirdinstance of the input data, every tenth instance of the input data, andso on.

In some examples, the motifs may be selected from patterns encounteredby the DNN during training. For example, the DNN may be trained forobject detection, including outputting predictions corresponding topredicted locations of objects detected in sensor data. In such anexample, the motifs may be selected to match objects encountered by theneural network during training in the training dataset. In this way, theDNN's training may be leveraged to determine whether the network hastransient faults, and the DNN may not need to be retrained to detect themotifs. Further, the location, number, class, and/or type of motifs maybe changed at different iterations of the input data, thereby creating arobust fault detection system with increased fault coverage.

Now with reference to FIG. 1 , FIG. 1 is an example data flow diagramillustrating an example process 100 for fault detection in neuralnetworks using motifs and/or watermarks, in accordance with someembodiments of the present disclosure. While the detection typesprimarily described herein with respect to FIG. 1 are transient orpermanent fault detections, this is not intended to be limiting, and isfor example purposes only. For example, the process 100 may be used todetect and classify any number of attributes and/or causes of thefaults, such as those described herein, without departing from the scopeof the present disclosure.

The process 100 may include generating and/or receiving sensor data 102from one or more sensors. The sensor data 102 may be received, as anon-limiting example, from one or more sensors of a vehicle (e.g.,vehicle 1000 of FIGS. 10A-10C and described herein). The sensor data 102may be used by the vehicle, and within the process 100, to detect andclassify neural network faults in real-time or near real-time. Thesensor data 102 may include, without limitation, sensor data 102 fromany of the sensors of the vehicle including, for example and withreference to FIGS. 10A-10C, global navigation satellite systems (GNSS)sensor(s) 1058 (e.g., Global Positioning System sensor(s)), RADARsensor(s) 1060, ultrasonic sensor(s) 1062, LIDAR sensor(s) 1064,inertial measurement unit (IMU) sensor(s) 1066 (e.g., accelerometer(s),gyroscope(s), magnetic compass(es), magnetometer(s), etc.),microphone(s) 1076, stereo camera(s) 1068, wide-view camera(s) 1070(e.g., fisheye cameras), infrared camera(s) 1072, surround camera(s)1074 (e.g., 360 degree cameras), long-range and/or mid-range camera(s)1078, speed sensor(s) 1044 (e.g., for measuring the speed of the vehicle1000), and/or other sensor types. As another example, the sensor data102 may include virtual sensor data generated from any number of sensorsof a virtual vehicle or other virtual object. In such an example, thevirtual sensors may correspond to a virtual vehicle or other virtualobject in a simulated environment (e.g., used for testing, training,and/or validating neural network performance), and the virtual sensordata may represent sensor data captured by the virtual sensors withinthe simulated or virtual environment. As such, by using the virtualsensor data, the machine learning model(s) 108 described herein may betested, trained, and/or validated using simulated data in a simulatedenvironment, which may allow for testing more extreme scenarios outsideof a real-world environment where such tests may be less safe.

In some embodiments, the sensor data 102 may include image datarepresenting an image(s), image data representing a video (e.g.,snapshots of video), and/or sensor data representing representations ofsensory fields of sensors (e.g., depth maps for LIDAR sensors, a valuegraph for ultrasonic sensors, etc.). Where the sensor data 102 includesimage data, any type of image data format may be used, such as, forexample and without limitation, compressed images such as in JointPhotographic Experts Group (JPEG) or Luminance/Chrominance (YUV)formats, compressed images as frames stemming from a compressed videoformat such as H.264/Advanced Video Coding (AVC) or H.265/HighEfficiency Video Coding (HEVC), raw images such as originating from RedClear Blue (RCCB), Red Clear (RCCC), or other type of imaging sensor,and/or other formats. In addition, in some examples, the sensor data 102may be used within the process 100 without any pre-processing (e.g., ina raw or captured format), while in other examples, the sensor data 102may undergo pre-processing (e.g., noise balancing, demosaicing, scaling,cropping, augmentation, white balancing, tone curve adjustment, etc.,such as using a sensor data pre-processor (not shown)). As used herein,the sensor data 102 may reference unprocessed sensor data, pre-processedsensor data, or a combination thereof.

Motif(s) 104 may include, without limitation, motifs and/or watermarksto be included with (e.g., appended) the sensor data 102 to generatecombined sensor data 106. In some embodiments, the motif(s) 104 may berepresented by image data corresponding to the types of predictions themachine learning model(s) 108 is trained to make. For example, for amachine learning model(s) 108 trained to detect and/or classifyobject(s) such as cars, trucks, road signs, pedestrians, etc., themotif(s) 104 may be chosen to be an image of a car, truck, road sign,and/or pedestrian. In another example, for a machine learning model(s)108 trained to detect and/or classify lanes, the motif(s) 104 may bechosen as an additional line(s) or lane marking(s) of a lane. As such,in some examples, the motif(s) 104 may be representative of patternsalready encountered by the machine learning model(s) 108 during trainingand/or inference. In such an example, the motif(s) 104 may be selectedto match objects encountered by the machine learning model(s) 108 duringtraining in a training dataset. In this way, the training of the machinelearning model(s) 108 may be leveraged to determine whether the networkhas transient faults, and the machine learning model(s) 108 may not needto be retrained, or separately trained, to detect the motif(s) 104.

In other examples, the machine learning model(s) 108 may be trainedusing training sensor data that is similar to the combined sensor data106. For example, during training, the training sensor data may beappended or augmented with motif(s) 104 such that the machine learningmodel(s) 108 are trained to predict the motif(s) 104 directly. In suchexamples, the machine learning model(s) 108 may be trained for a firsttask(s) with respect to the training sensor data and a second task(s)with respect to the motif(s) 104. As a non-limiting example, the machinelearning model(s) 108 may be trained on depth maps corresponding toLIDAR data, and the depth maps may be appended with a motif(s) 104 ofobjects to be detected. As such, the machine learning model(s) 108 maybe trained to make predictions using the depth maps while also beingtrained to make different predictions corresponding to the motif(s) 104.In other embodiments, the use of the motif(s) 104 in training may be forthe same task as the primary task of the machine learning model(s) 108(e.g., object detection).

The combined sensor data 106 may include data representative of acombination of the sensor data 102 and the motif(s) 104. The combinedsensor data 106 may be used as an input to the machine learning model(s)108. In some examples, a similar type, number, and/or class of motif(s)104, or the exact type, number, and/or class of motif(s) 104, may beincluded in the combined sensor data 106 at each iteration. In otherexamples, different types, number, or classes of motif(s) 104 may beintroduced to the sensor data 102 at each iteration. In addition, alocation and/or size of the motif(s) 104 may remain constant acrossiterations or may change at different iterations. In some embodiments,to increase fault coverage, the locations, types, number, classes,and/or sizes of the motif(s) 104 may be changed at different iterationsof the combined sensor data 106. For example, because different portionsof the underlying hardware and/or software processing the machinelearning model(s) 108 may process information corresponding to differentportions of the combined sensor data 106, by moving, resizing, orotherwise changing the motif(s) 104 in different iterations, a greaterextent of the underlying hardware and/or software may be fault tested.

In non-limiting embodiments, the machine learning model(s) 108 may betrained using training data (e.g., training images or other trainingdata representations) corresponding to a full image or otherrepresentation (e.g., without motif(s) 104) at a first spatialresolution. In such embodiments, during inference, the sensor data102—or the image or other representation thereof—may be downscaled fromthe first spatial resolution to a second spatial resolution, and themotif(s) 104 may be appended to the sensor data 102 to generate thecombined sensor data 106 at the first spatial resolution. As such, thetraining sensor data may be at a same spatial resolution as the combinedsensor data 106 that includes the motif(s) 104, thereby allowing themachine learning model(s) 108 to be trained for a downstream task (e.g.,object detection, line regression, etc.) using training sensor data thatis different (e.g., doesn't have motif(s) 104) from the combined sensordata 106 used during inference. In such embodiments, the motif(s) 104may represent, as described herein, representations that the machinelearning model(s) 108 is trained to detect or regress on—such as certaintypes of objects or other features of environments. As such, a machinelearning model(s) 108 trained to perform a particular task without faultdetection in mind, may then have the fault detection methods describedherein (e.g., appending or otherwise adding motif(s) 104 to sensor data102) implemented at inference without impacting the performance of themachine learning model(s) 108 or requiring retraining. As non-limitingexamples, FIGS. 3A-3B illustrate—among other things—how an originalimage may be downsized and have a motif(s) 104 appended thereto togenerate combined sensor data 106.

The location, class, number, size, and/or type of motif(s) 104 from thecombined sensor data 106 may be tracked and/or stored as expectedprediction(s) 112. Where the motif(s) 104 is fixed at each iteration,the expected predictions(s) 112 may also be fixed, and only stored asingle time. Where the motif(s) 104 is dynamic, the location, class,number, size, and/or type of motif(s) 104 may be tracked over time andstored for use by a fault detector 114 in determining whether or not afault is present. For example, the fault detector 114 may compare datafrom an output(s) 110 to the expected prediction(s) 112 to see whetherthe output(s) 110 aligns with the expected prediction(s) 112 (e.g., if amotif is a vehicle of class A in an upper right of an image, the faultdetector 114 may look for data in the output(s) 110 indicating a vehicleis present, of class A, in the upper right of an image—as denoted by alocation of a bounding box, for example). In some examples, the expectedprediction(s) 112 may be stored as a data structure storing pixellocations where each motif(s) 104 is located and/or a corresponding typeand/or class of the motif(s) 104.

The machine learning model(s) 108 may use as input one or more images orother data representations (e.g., depth maps or point clouds from LIDARdata, images from RADAR data, etc.) as represented by the combinedsensor data 106 to generate output(s) 110. In a non-limiting example,the machine learning model(s) 108 may take, as input, an image(s)represented by the combined sensor data 106 (e.g., after appending thesensor data 102 with motif(s) 104) to generate the output(s) 110.Although examples are described herein with respect to using neuralnetworks, and specifically DNNs, as the machine learning model(s) 108,this is not intended to be limiting. For example, and withoutlimitation, the machine learning model(s) 108 described herein mayinclude any type of machine learning model, such as a machine learningmodel(s) using linear regression, logistic regression, decision trees,support vector machines (SVM), Naïve Bayes, k-nearest neighbor (Knn), Kmeans clustering, random forest, dimensionality reduction algorithms,gradient boosting algorithms, neural networks (e.g., auto-encoders,convolutional, recurrent, perceptrons, Long/Short Term Memory (LSTM),Hopfield, Boltzmann, deep belief, deconvolutional, generativeadversarial, liquid state machine, etc.), and/or other types of machinelearning models. Further, the machine learning model(s) 108 may be anymachine learning model trained to detect any types and/or classes ofobjects, lines, etc. that the DNN is trained to predict. As such, themachine learning model(s) 108 may be an object detection model,segmentation model, lane detection model, etc.

The output(s) 110 of the machine learning model(s) 108 may includepredictions corresponding to the combined sensor data 106. For example,the output(s) of the machine learning model(s) may include types,classes, sizes, locations, values, and/or other informationcorresponding to the combined sensor data 106. For example, for amachine learning model(s) 108 trained to detect objects, the output(s)110 may include pixel locations for vertices of bounding shapescorresponding to the objects and/or class labels associated therewith.In some embodiments, the outputs of the machine learning model(s) 108for one or more of the predictions may include confidence values. Forexample, each pixel in a predicted pattern region, or for each predictedpattern (or vertices thereof), confidence values may be output for eachclass type the machine learning model(s) 108 is trained to predict, withthe highest confidence being determined to be the class of the object.As such, once the highest confidence class is determined, this class maybe compared to the expected prediction(s) 112 to determine whether ornot there is an issue—and ultimately whether or not there is a fault.

To determine the faults, as described herein, the output(s) 110 of themachine learning model(s) 108 and the expected prediction(s) 112 may beanalyzed or compared by the fault detector 114 to determine the accuracyof the output(s) 110 with respect to the expected prediction(s) 112determined from the combined sensor data 106. As another non-limitingexample with respect to object detection, for each motif of the motif(s)104 introduced to the sensor data 102, the expected prediction(s) 112including the actual location and/or actual class label with respect tothe combined sensor data 106 may be compared with the predictedlocations and/or the predicted class labels in the output(s) 110 tocheck the accuracy of the predictions or results.

Based on the comparison, the accuracy or resiliency of the machinelearning model(s) 108 may be determined, such as by identifying if thepredictions or output(s) 110 of the machine learning model(s) 108 do notcorrespond to the motif information known to the fault detector 114, orthe predictions are outside of a threshold accuracy with respect to themotif information. In some examples, when the expected predictions(s)112 for each motif(s) 104 are represented in the output(s) 110, thefault detector 114 may determine that the predictions of the machinelearning model(s) 108 are accurate and the underlying hardware and/orsoftware of the machine learning model(s) 108 does not have a fault.Similarly, when the expected predictions(s) 112 for each motif(s) 104are not represented in the output(s) 110, the fault detector 114 maydetermine that the predictions of the machine learning model(s) 108 areinaccurate and the underlying hardware and/or software of the machinelearning model(s) 108 has a fault(s). In an example, output(s) 110 mustinclude representation of each of the motif(s) 104 in the combinedsensor data 106 for the machine learning model(s) 108 to be determinedfault-free. The machine learning model(s) 108 may be determined to havea transient fault when the results of the machine learning model(s) 108are found to be inaccurate. This may allow the system to perform faultdetection using a single instance of the machine learning model(s) 108while—in some embodiments—leveraging the training of the machinelearning model(s) 108 to detect faults in real-time or near real-time.

Once a fault or no-fault determination is made, this information may bepassed to one or more components of the system to make controldecision(s) 116. For example, where the system is the vehicle 1000,described herein, the fault determination may be passed to one or morelayers of an autonomous driving software stack (e.g., a planning layer,a control layer, a world-model manager, a perception layer, an obstacleavoidance layer of the drive stack, an actuation layer of the drivestack, etc.) to determine an appropriate control decision(s) 116. Forexample, where predictions are outside of a threshold accuracy withrespect to the motif information, some or all of the outputs of themachine learning model(s) 108 may be skipped over or disregarded withrespect to one or more of the control decision(s) 116. In some examples,such as where the predictions of the machine learning model(s) 108 areinaccurate and unusable for safe operation of the vehicle 1000, thecontrol decision(s) 116 may include handing control back to a driver(e.g., exiting autonomous or semi-autonomous operation), or executing anemergency or safety maneuver (e.g., coming to a stop, pulling to theside of the road, or a combination thereof). As such, the controldecision(s) 116 may include suggesting one or more corrective measuresfor effective and safe driving—such as ignoring certain results of themachine learning model(s) 108. In any example, and with respect toautonomous or semi-autonomous driving, the control decision(s) 116 mayinclude any decisions corresponding to a neural network manager layer ofan autonomous driving software stack (alternatively referred to hereinas a “drive stack”), a perception layer of the drive stack, a worldmodel management layer of the drive stack, a planning layer of the drivestack, a control layer of the drive stack, an obstacle avoidance layerof the drive stack, and/or an actuation layer of the drive stack. Inexamples where there are no faults detected, the control decision(s) 116may still be impacted by this information. For example, certainfunctions or features of the autonomous driving software stack may notbe relied upon or used without an indication or signal being received(e.g., at each iteration, at an interval, etc.) indicating that themachine learning model(s) 108 is working properly. In some examples, theprocess 100 may be executed on any number of machine learning model(s)108 operating within a system. For example, an autonomous drivingsoftware stack may rely on hundreds or thousands of machine learningmodel(s) 108 for effective and safe operation, and any number of thesemay be subject to the process 100 in order to ensure faults do notinterfere with the safe and effective operation. As such, as describedherein, the accuracy of the output(s) 110 may be separately determinedfor any number of different operations corresponding to one or morelayers of the drive stack, using one or more machine learning model(s)108. As an example, a first accuracy may be determined for objectdetection operations with respect to the perception layer of the drivestack using a first machine learning model, and a second accuracy may bedetermined for path planning with respect to the planning layer of thedrive stack using a second machine learning model trained for regressingon lane lines.

Now referring to FIG. 2 , FIG. 2 is an illustration of an examplemachine learning model(s) 108 trained to detect objects, in accordancewith some embodiments of the present disclosure. As described herein,machine learning model(s) 108 for object detection is only for examplepurposes, and other types of machine learning model(s) 108 may be usedwithout departing from the scope of the present disclosure. In addition,although vehicles, trees, and buildings are primarily illustrated asexamples of objects for detection, this is not intended to be limiting.As additional non-limiting examples, the machine learning model(s) 108may be trained to perform inferencing for forecast predictions usingimages of clouds or cloud structures, for depth predictions using images(even though trained using a combination of LIDAR data, RADAR data,and/or image data, for example), for predicting conditions of offshoreoil or gas pipelines using sensor data, etc.

With respect to FIG. 2 , for example, the machine learning model(s) 108may be trained to take as input the sensor data 102 representative of animage 202, and may output predictions 208 corresponding to objects inthe image 202. The predictions 208 may be generated based on the objectlocation(s) and/or associated object classifications (e.g., classlabels) computed by the machine learning model(s) 108. For example,based on the shape, size, and/or location of the objects detected, eachobject may be classified as a car, bus, stop sign, pedestrian, truck,tree, building, etc.—e.g., based on the number of classes the machinelearning model(s) 108 has been trained to detect. The machine learningmodel(s) 108 may output pixel by pixel output data 206 with each pixelassociated with a value based on the predicted object classification (orempty class), or may output regressed pixel locations corresponding tothe object(s) or bounding shapes corresponding thereto.

With reference to FIGS. 3A-3B, FIGS. 3A-3B are example illustrations ofappending motifs to an input image, in accordance with some embodimentsof the present disclosure. For example, the machine learning model(s)108 may use image data representative of a combined image 310—thatincludes data representative of an input image 302 and one or moremotifs 304—as input to generate predictions for the combined image 310.Referring to FIG. 3A, input image 302 may be introduced with motifs 304(e.g., motifs 304A, 304B, 304C) to generate a combined image 310 withthe motifs 304 appended on the input image 302. As described herein, insome examples the input image may be downsampled (e.g., by 10%, 20%,35%, etc.) to generate a downsampled image 306 and the motifs 304 may beintroduced to the downsampled image 306. In some embodiments, the motifs304 may be located within the downsampled image 306 such that they arelocated outside the boundary of the downsampled image 306 whilemaintaining a spatial resolution for the combined image 310 that is thesame or less than a spatial resolution of the original input image 302.For example, the motif 304A, the motif 304B, and the motif 304C may beadded along a boundary (e.g., a top boundary, a bottom boundary, a sideboundary, or a combination thereof) outside of the downsampled image306, where the motifs 304 are sized such that the spatial resolution ofthe combined image 310 is similar to or the same as the original inputimage 302 (e.g., a spatial resolution that the machine learning model(s)108 was trained on, and is programmed for).

Referring now to FIG. 3B, FIG. 3B illustrates an example illustration ofappending motifs to an image captured by an image sensor of a vehicle.For example, the machine learning model(s) 108 may use image datarepresentative of a combined image 330—that includes data representativeof an input image 322 captured by the image sensor of the vehicle 1000and one or more motifs 324—to generate predictions for the combinedimage 330. Similar to the description above with respect to the combinedimage 310, the input image 322 may be introduced with the motifs 324(e.g., the motifs 324A, 324B, 324C) to generate the combined image 330.

Although FIGS. 3A-3B include appending the motifs 304 and 324 to aboundary of the original input images 302 and 322, respectively, this isnot intended to be limiting. For example, the motifs may be added withinthe original images, may be added outside of boundaries of the originalimages, or a combination thereof. In addition, although the originalinput images 302 and 322 are described as being downsampled prior toappending the motifs 304 and 324, respectively, this is not intended tobe limiting. For example, during training and inference the size of theoriginal images may remain the same, and the motifs may either be addedwithin the original images or the images used during training may alsoinclude portions designated for motifs (e.g., the machine learningmodel(s) 108 may be trained on training data that also includes themotifs, or that at least includes additional portions where motifs maybe inserted).

Now referring to FIG. 4 , FIG. 4 is an example illustration of a processfor generating outputs using a machine learning model with respect tofault detection, in accordance with some embodiments of the presentdisclosure. The process 400 may include a visual representation of aninstance of the process 100 of FIG. 1 that may be used to apply imagedata introduced with motifs to the machine learning model(s) 108 togenerate outputs corresponding to the original image data and theintroduced motifs. As a non-limiting example, the machine learningmodel(s) 108 in process 400 may be trained for object detection—andspecifically for detecting cars, trucks, and trees. The machine learningmodel(s) 108, during inference, may take as input instances of thecombined sensor data 106 corresponding to the sensor data 102 (e.g.,image data) representative of an image captured by a sensor of avehicle—such as the vehicle 1000—combined with motif(s) 104A, 104B, and104C. The machine learning model(s) 108 may then generate outputs 406(e.g., pixel by pixel output data with each pixel associated with avalue based on the predicted object classification (or empty class), thepixels corresponding to positions or locations with respect to thecombined sensor data 106) that may correspond to predictions 410Arelated to the sensor data 102 and predictions 410B related to themotif(s) 104. The output(s) 110 may include object information (e.g.,bounding boxes, class labels, locations) for regions where each objectis predicted to be located in the combined sensor data 106.

Although described as separate predictions 410A and 410B, this is notintended to be limiting. In practice, the predictions 410A and 410B maycorrespond to a single set of predictions, and the fault detector 114may determine whether the predictions include a subset of predictions410B corresponding to the motif(s) 104, and whether those predictions410B are accurate.

Based on the locations (e.g., regions, pixels) where each object isdetected, in some examples, the predictions 410B for motif(s) 104 may becompared to the known or actual locations of the motif(s) 104 in thecombined sensor data 106 to determine the accuracy of the machinelearning model(s) 108 and thus whether or not a fault is present. Forexample, and with reference to FIG. 5 , predictions 410B of the machinelearning model(s) 108 may be compared against the expected prediction(s)112 to determine accuracy of the results of the machine learningmodel(s) 108. A fault in the machine learning model(s) 108 may bedetected at comparison block B512 based on a one-to-one comparisonbetween the expected prediction(s) 112 and the prediction(s) 410B forthe motif(s) 104 output by the machine learning model(s) 108. In thenon-limiting examples of FIGS. 4 and 5 , the expected prediction(s) 112may include class labels and/or locations of each motif(s) 104 in thecombined sensor data 106. The expected prediction(s) 112 for themotif(s) 104A, 104B, and 104C may have been stored when the motif(s)104A, 104B, and 104C are appended to or otherwise added to the sensordata 102 to generate the combined sensor data 106.

The expected prediction(s) 112 for each motif may be compared againstthe predictions 410B of the machine learning model(s) 108. If there is adifference (e.g., in location, class label, etc.) between the expectedprediction(s) 112 and the prediction(s) 410B with respect to any of themotif(s) 104A, 104B, and 104C, the system may determine that a potentialfault is detected (at block B514A) in the machine learning model(s) 108.If the expected prediction(s) 112 matches the prediction(s) 410B foreach of the motif(s) 104A, 104B, and 104C, the system may determine thatthere is no fault (at block B514B) detected in the machine learningmodel(s) 108. In other examples, the machine learning model(s) 108 maybe determined to have a transient fault when an accuracy error isdetected (at block B514A). Further, control decisions may be made basedon whether or not a fault is detected in the machine learning model(s)108. In some examples, for each instance of the combined sensor data 106where no fault is determined, the predictions 410A may be relied upon bythe underlying system.

Now referring to FIG. 6 , each block of method 600, described herein,comprises a computing process that may be performed using anycombination of hardware, firmware, and/or software. For instance,various functions may be carried out by a processor executinginstructions stored in memory. The method 600 may also be embodied ascomputer-usable instructions stored on computer storage media. Themethod 600 may be provided by a standalone application, a service orhosted service (standalone or in combination with another hostedservice), or a plug-in to another product, to name a few. In addition,method 600 is described, by way of example, with respect to the process100 of FIG. 1 . However, this method 600 may additionally oralternatively be executed by any one system or within any one process,or any combination of systems and processes, including, but not limitedto, those described herein.

FIG. 6 is a flow diagram showing a method 600 for detecting faults in aneural network using motifs and/or watermarks, in accordance with someembodiments of the present disclosure. The method 600, at block B602,includes generating first data representative of sensor data captured bya sensor and one or more motifs appended to the sensor data. Forexample, first data (e.g., the combined sensor data 106) may begenerated that represents sensor data (e.g., the sensor data 102) andmotifs (e.g., the motif(s) 104) appended to the sensor data.

The method 600, at block B604, includes applying the first data to aneural network. For example, the first data (e.g., the combined sensordata 106) may be applied (e.g., as an input) to the machine learningmodel(s) 108 (e.g., a neural network, such as a DNN).

The method 600, at block B606, includes computing, using the neuralnetwork, second data representative of predictions of the neural networkwith respect to the first data. For example, the machine learningmodel(s) 108 may compute the output(s) 110 (e.g., class labels,locations, pixels, values, confidences, etc.) associated withpredictions of the machine learning model(s) 108.

The method 600, at block B608, includes comparing the second data tothird data, the third data representative of expected predictionscorresponding to the one or more motifs represented by the first data.For example, the output(s) 110 of the machine learning model(s) 108 maybe compared to the expected prediction(s) 112 (e.g., actual locationsand/or class labels of the motif(s) 104) by the fault detector 114. Thefault detector 114 may compare the output(s) 110 of the machine learningmodel(s) 108 against the expected prediction(s) 112 for each of the oneor more motif(s) 104, for one or more instances of the combined sensordata 106.

The method 600, at block B610, includes determining that the predictionsare accurate when, based at least in part on the comparing, the expectedpredictions are represented in the predictions of the neural network.For example, the fault detector 114 may determine that the predictions(e.g., the output(s) 110) are accurate when the expected prediction(s)112 are represented in the output(s) 110 of the machine learningmodel(s) 108. The fault detector 114 may determine that the expectedprediction(s) 112 are represented in the output(s) 110 when expectedprediction(s) 112 for each of the one or more motif(s) 104 is found inthe output(s) 110, or when a threshold similarity is determined (e.g.,for object detection, so long as predicted locations of vertices of abounding shape are within 95% accuracy, 98% accuracy, etc., with respectto the expected prediction(s) 112).

The method 600, at block B612, includes determining that the predictionsare inaccurate when, based at least in part on the comparing, theexpected predictions are not represented in the predictions of theneural network. For example, the fault detector 114 may determine thatthe predictions (e.g., the output(s) 110) are inaccurate when theexpected prediction(s) 112 are not represented in the output(s) 110 ofthe machine learning model(s) 108. The fault detector 114 may determinethat the expected prediction(s) 112 are not represented in the output(s)110 when the expected prediction(s) 112 for at least one of the one ormore motif(s) 104 is not found in the output(s) 110, or is outside of athreshold similarity to the expected prediction(s) 112.

Permanent Fault Detection Method

For detecting permanent faults, input data representative of a signatureimage may be provided as input to a deep neural network (DNN), where theDNN is either trained to predict a certain signature with respect to thesignature image, or is trained to make predictions for images similar tothe signature image, such that an expected signature is known. In suchembodiments, the signature image may be applied to the DNNintermittently, and the predicted signature may be compared against theexpected signature to determine whether a fault is detected.

In embodiments, to detect permanent fault(s) in a DNN, sensor data(e.g., images, videos, depth maps, etc.) may be received from sensors(e.g., cameras, LIDAR sensors, RADAR sensors, etc.). In non-limitingexamples corresponding to autonomous vehicles, the sensors may bedisposed on or otherwise associated with an autonomous orsemi-autonomous vehicle. The sensor data captured may be applied to aDNN—e.g., as a sequence of images, as a sequence of depth maps, etc. Insome embodiments, intermittently, while applying the sensor data to theneural network, a signature image (e.g., an image including motifs, astock image, a logo, etc.) may be applied to the DNN. For example, thesignature image may be applied at an interval, every x number of images,randomly, and/or the like. At an iteration where a signature image isapplied, the actual predictions of the DNN may be compared to anexpected prediction of the DNN with respect to the signature image.

Accuracy of the predictions of the DNN may be determined based on thecomparison of the expected prediction to the actual prediction. Forexample, the expected prediction may include a signature, which may beknown to a fault detector, and the output signature of the DNN may becompared to this signature. In some examples, a threshold may be usedsuch that predictions of the DNN with respect the signature image shouldbe within a threshold similarity to the expected predictions. When theoutput of the DNN is determined to be inaccurate, hardware and/orsoftware corresponding to processing of the DNN may be considered tohave a permanent fault. In such examples, corrective measures may betaken, such as to hand over the control of the vehicle to the driver,perform a safety maneuver, and/or to offload or transfer processing ofthe DNN to other components—e.g., a redundant architecture.

With reference to FIG. 7 , FIG. 7 is an example data flow diagramillustrating a process 700 for detecting faults in a neural networkusing signatures, in accordance with some embodiments of the presentdisclosure. The process 700 may be used for detecting permanent faultsin any machine learning model(s) 108, such as but not limited to thosedescribed herein. In addition, as described herein, the process 700 isnot limited to detecting only permanent faults, and may be used todetect other types of faults without departing from the scope of thepresent disclosure—such as transient or intermittent faults. The machinelearning model(s) 108 may receive, as input, the sensor data 102, whichmay be similar to the sensor data 102 described herein. Although thesensor data 102 is primarily discussed with respect to image datarepresentative of image(s), this is not intended to be limiting. In someembodiments, for example, the sensor data 102 may include data from oneor more LIDAR sensors, RADAR sensors, SONAR sensors, ultrasonic sensors,IMU sensors, and/or other sensor types (e.g., sensors and/or camerasdescribed with respect to FIGS. 10A-0C).

The sensor data 102 used as input for the machine learning model(s) 108may include original images (e.g., as captured by one or more imagesensors), down-sampled images, up-sampled images, cropped or region ofinterest (ROI) images, labeled images annotated for or outputted fromother systems, flipped on an axis or otherwise augmented images, or acombination thereof. The sensor data 102 may represent, in somenon-limiting embodiments, images captured by one or more sensors (e.g.,cameras of a vehicle 1000), and/or may represent images captured fromwithin a virtual environment used for testing and/or generating trainingimages (e.g., a virtual camera of a virtual vehicle within a virtual orsimulated environment). In some examples, the sensor data 102 may beapplied to the machine learning model(s) 108 as a sequence of images, asequence of depth maps, and/or a sequence of another sensor datarepresentation.

Intermittently, while applying the sensor data 102 to the machinelearning model(s), signature data 704 may be applied to the machinelearning model(s) 108. The signature data 704 may be representative of asignature image (e.g., an image including motifs, a stock image, a logo,etc.) that may be entirely different from or may be similar to thepatterns (e.g., objects, lanes, lines, etc.) that the machine learningmodel(s) 108 is trained to predict. The signature data 704 may beapplied to the machine learning model(s) 108 at a fixed or variedinterval within the sequence of the sensor data 102. For example, thesignature data 704 may be applied at an interval, every x number ofimages, randomly, and/or the like. The signature data 704 may includeone or more signature images. At each iteration, the same or a differentsignature image may be applied to the machine learning model(s) 108. Forexample, where different signature images are applied, any number ofsignature images and corresponding expected prediction(s) 112 may bestored, and the signature images may be applied to the machine learningmodel(s) 108 in order, randomly, and/or the like.

The machine learning model(s) 108 may be any machine learning model(e.g., neural network, DNN, CNN, etc.) that is trained to makepredictions based on sensor data 102 (or other data types, as describedherein). The machine learning model(s) 108 may be configured to generateoutput(s) 110 based on the sensor data 102 and the signature data704—separately, in embodiments. For example, in some embodiments, themachine learning model(s) 108 may be trained to predict a certain (e.g.,expected) signature with respect to a signature image(s) of thesignature data 704 and may be trained to compute different predictionswith respect to the sensor data 102. In such examples, training themachine learning model(s) 108 may include using a first loss functionwith respect to the signature data 704 and a second loss function withrespect to the sensor data 102. In other examples, the machine learningmodel(s) 108 may be trained to predict certain signature(s) for imagessimilar to a signature image of the signature data 704. In bothexamples, the expected prediction(s) 112 may be stored for the signaturedata 704 such that the fault detector 114 may compare the output(s) 110to the expected prediction(s) 112. In some examples, a placement (e.g.,an image number, an interval of images, etc.) of the signature image(s)within the sequence of the sensor data 102 may also be stored to aid thefault detector 114 in determining which output(s) 110 correspond to thesignature data 704.

At instances where the signature data 704 is applied, the machinelearning model(s) 108 may take as input the signature data 704 andgenerate the output(s) 110 corresponding thereto. These output(s) 110may be compared by the fault detector 114 with the expectedprediction(s) 112 for the signature data 704 to determine whether afault is present. The fault detector 114 may, in some non-limitingembodiments, use information corresponding to the signature data 704(e.g., placement within the sequence, type of signature expected to beoutput, etc.) to check against the output(s) 110 computed by the machinelearning model(s) 108. The comparison may include comparing expectedsignatures to predicted signatures to ensure that the output(s) 110 atleast include accurate (e.g., within a threshold) prediction(s)(e.g., ascompared to expected prediction(s) 112) with respect to the signaturedata 704.

Based on the comparison, the accuracy or resiliency of the machinelearning model(s) 108 may be determined, such as by identifying if thepredictions or output(s) 110 of the machine learning model(s) 108 do notcorrespond to the expected signatures known to the fault detector 114,or the predictions are outside of a threshold accuracy with respect tothe expected signature information. In some examples, when the expectedpredictions(s) 112 for each iteration of signature data 704 isrepresented in the output(s) 110, the fault detector 114 may determinethat the predictions of the machine learning model(s) 108 are accurateand the underlying software and/or hardware corresponding to the machinelearning model(s) 108 does not have a fault. Similarly, when theexpected predictions(s) 112 for each iteration of signature data 704 isnot represented in the output(s) 110, the fault detector 114 maydetermine that the predictions of the machine learning model(s) 108 areinaccurate and that the underlying software and/or hardwarecorresponding to the machine learning model(s) 108 has a fault(s).

Once a fault or no-fault determination is made, this information may bepassed to one or more components of the system to make controldecision(s) 116. For example, where the system is the vehicle 1000,described herein, the fault determination may be passed to one or morelayers of an autonomous driving software stack to determine anappropriate control decision(s) 116. For example, where predictions areoutside of a threshold accuracy with respect to the signature data 704,some or all of the output(s) 110 of the machine learning model(s) 108may be skipped over or disregarded with respect to one or more of thecontrol decision(s) 116. In some examples, such as where the predictionsof the machine learning model(s) 108 are inaccurate and unusable forsafe operation of the vehicle 1000, the control decision(s) 116 mayinclude handing control back to a driver (e.g., exiting autonomous orsemi-autonomous operation), or executing an emergency or safety maneuver(e.g., coming to a stop, pulling to the side of the road, or acombination thereof). Because the type of fault may be a permanentfault, an indicator or signal may be generated that indicatesdiagnostics or testing need to be performed before further operation—atleast with respect to the functionality(ies) that the machine learningmodel(s) 108 influences—is allowed. In examples where there are nofaults detected, the control decision(s) 116 may still be impacted bythis information. For example, certain functions or features of theautonomous driving software stack may not be relied upon or used withoutan indication or signal being received (e.g., at each iteration, at aninterval, etc.) indicating that the machine learning model(s) 108 isworking properly. In some examples, the process 100 may be executed onany number of machine learning model(s) 108 operating within a system.For example, an autonomous driving software stack may rely on hundredsor thousands of machine learning model(s) 108 for effective and safeoperation, and any number of these may be subject to the process 700 inorder to ensure faults do not interfere with the safe and effectiveoperation. As such, as described herein, the accuracy of the output(s)110 may be separately determined for any number of different operationscorresponding to one or more layers of the drive stack, using one ormore machine learning model(s) 108.

Now referring to FIGS. 8A-8C, FIGS. 8A-8C are example illustrations ofsignature inputs to determine faults in neural networks, in accordancewith some embodiments of the present disclosure. For example, thesignature images (e.g., signature images 810, 820, and/or 830) may beapplied—intermittently, in embodiments—to the machine learning model(s)108 in place of the sensor data 102 to test for faults in the underlyingsoftware and/or hardware executing the machine learning model(s) 108.For example, the signature image(s) may be applied to the machinelearning model(s) 108 at an interval, every x number of images,randomly, and/or the like. At an iteration where a signature image isapplied, the actual predictions of the DNN may be compared to anexpected prediction of the DNN with respect to the signature image.

The machine learning model(s) may be either trained to predict thesignatures with respect to the signature images, or may be trained tomake predictions for images similar to the signature images, such thatan expected signature is known. The machine learning model(s) 108 may betrained to make predictions for any number of signature images. Fornon-limiting examples, the machine learning model(s) 108 may be trainedto predict a signature with respect to the image 810 of FIG. 8A, or maybe trained to predict signatures with respect to each of the images 810,820, or 830. Where more than one signature image type is used, the faultcoverage may be increased as different portions of the underlyingsoftware and/or hardware may be involved in the processing with respectto different signature images. The images 810, 820, and 830 are merelyexamples, and are not intended to be limiting. The signature images mayinclude any image type, or may include another type of sensor datarepresentation or other data type representation (e.g., financial data).

Now referring to FIG. 9 , each block of method 900, described herein,comprises a computing process that may be performed using anycombination of hardware, firmware, and/or software. For instance,various functions may be carried out by a processor executinginstructions stored in memory. The method 900 may also be embodied ascomputer-usable instructions stored on computer storage media. Themethod 900 may be provided by a standalone application, a service orhosted service (standalone or in combination with another hostedservice), or a plug-in to another product, to name a few. In addition,method 900 is described, by way of example, with respect to the process700 of FIG. 7 . However, this method 900 may additionally oralternatively be executed by any one system or within any one process,or any combination of systems or processes, including, but not limitedto, those described herein.

FIG. 9 is a flow diagram showing a method 900 for detecting faults in aneural network using signature images, in accordance with someembodiments of the present disclosure. The method 900, at block B902,includes applying first data representative of a plurality of sensordata instances captured by a sensor to a neural network. For example,first data that represents sensor data (e.g., the sensor data 102) maybe applied (e.g., as an input) to the machine learning model(s) 108(e.g., a neural network, such as a DNN).

The method 900, at block B904, includes intermittently, during theapplying the first data to the neural network, applying second datarepresentative of a signature input to the neural network. For example,a signature input (e.g., the signature data 704) may be applied to themachine learning model(s) 108 intermittently while applying the sensordata 102 to the machine learning model(s) 108.

The method 900, at block B906, includes comparing third datarepresentative of an output of the neural network computed based atleast in part on the second data to fourth data representative of anexpected output of the neural network based at least in part on thesecond data. For example, the output(s) 110 of the machine learningmodel(s) 108 may be compared to the expected prediction(s) 112 (e.g.,expected output), the expected prediction(s) 112 including an expectedprediction for the signature input. The fault detector 114 may comparethe output(s) 110 against the expected prediction(s) 112 for thesignature data 704.

The method 900, at block B908, includes determining that predictions ofthe neural network are accurate when, based at least in part on thepredictions, the output corresponds to the expected output. For example,the fault detector 114 may determine that the predictions (e.g.,output(s) 110) of the machine learning model(s) 108 are accurate if theoutput(s) 110 corresponds to the expected prediction(s) 112. The faultdetector 114 may determine that the predictions are accurate when theoutput(s) 110 includes the expected prediction(s) 112 for the signaturedata 704.

The method 900, at block B910, includes determining that predictions ofthe neural network are inaccurate when, based at least in part on thepredictions, the output does not correspond to the expected output. Forexample, the fault detector 114 may determine that the predictions(e.g., output(s) 110) of the machine learning model(s) 108 areinaccurate if the output(s) 110 does not correspond to the expectedprediction(s) 112. The fault detector 114 may determine that thepredictions are inaccurate when the output(s) 110 does not include theexpected prediction(s) 112 for the signature data 704.

Example Autonomous Vehicle

FIG. 10A is an illustration of an example autonomous vehicle 1000, inaccordance with some embodiments of the present disclosure. As describedherein, in some non-limiting embodiments, such as where the machinelearning model(s) 108 are included within an autonomous driving softwarestack, the vehicle 1000 may represent the underlying system executingtesting of the machine learning model(s) 108. For example, the vehicle1000 may execute built-in self-test (BIST) during deployment, and thetesting of the machine learning model(s) 100 according to the processes100 and 700 may be included within the BIST of the vehicle 1000. Theautonomous vehicle 1000 (alternatively referred to herein as the“vehicle 1000”) may include, without limitation, a passenger vehicle,such as a car, a truck, a bus, a first responder vehicle, a shuttle, anelectric or motorized bicycle, a motorcycle, a fire truck, a policevehicle, an ambulance, a boat, a construction vehicle, an underwatercraft, a drone, and/or another type of vehicle (e.g., that is unmannedand/or that accommodates one or more passengers). Autonomous vehiclesare generally described in terms of automation levels, defined by theNational Highway Traffic Safety Administration (NHTSA), a division ofthe US Department of Transportation, and the Society of AutomotiveEngineers (SAE) “Taxonomy and Definitions for Terms Related to DrivingAutomation Systems for On-Road Motor Vehicles” (Standard No.J3016-201806, published on Jun. 15, 2018, Standard No. J3016-201609,published on Sep. 30, 2016, and previous and future versions of thisstandard). The vehicle 1000 may be capable of functionality inaccordance with one or more of Level 3-Level 5 of the autonomous drivinglevels. For example, the vehicle 1000 may be capable of conditionalautomation (Level 3), high automation (Level 4), and/or full automation(Level 5), depending on the embodiment.

The vehicle 1000 may include components such as a chassis, a vehiclebody, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and othercomponents of a vehicle. The vehicle 1000 may include a propulsionsystem 1050, such as an internal combustion engine, hybrid electricpower plant, an all-electric engine, and/or another propulsion systemtype. The propulsion system 1050 may be connected to a drive train ofthe vehicle 1000, which may include a transmission, to enable thepropulsion of the vehicle 1000. The propulsion system 1050 may becontrolled in response to receiving signals from thethrottle/accelerator 1052.

A steering system 1054, which may include a steering wheel, may be usedto steer the vehicle 1000 (e.g., along a desired path or route) when thepropulsion system 1050 is operating (e.g., when the vehicle is inmotion). The steering system 1054 may receive signals from a steeringactuator 1056. The steering wheel may be optional for full automation(Level 5) functionality.

The brake sensor system 1046 may be used to operate the vehicle brakesin response to receiving signals from the brake actuators 1048 and/orbrake sensors.

Controller(s) 1036, which may include one or more system on chips (SoCs)1004 (FIG. 10C) and/or GPU(s), may provide signals (e.g., representativeof commands) to one or more components and/or systems of the vehicle1000. For example, the controller(s) may send signals to operate thevehicle brakes via one or more brake actuators 1048, to operate thesteering system 1054 via one or more steering actuators 1056, to operatethe propulsion system 1050 via one or more throttle/accelerators 1052.The controller(s) 1036 may include one or more onboard (e.g.,integrated) computing devices (e.g., supercomputers) that process sensorsignals, and output operation commands (e.g., signals representingcommands) to enable autonomous driving and/or to assist a human driverin driving the vehicle 1000. The controller(s) 1036 may include a firstcontroller 1036 for autonomous driving functions, a second controller1036 for functional safety functions, a third controller 1036 forartificial intelligence functionality (e.g., computer vision), a fourthcontroller 1036 for infotainment functionality, a fifth controller 1036for redundancy in emergency conditions, and/or other controllers. Insome examples, a single controller 1036 may handle two or more of theabove functionalities, two or more controllers 1036 may handle a singlefunctionality, and/or any combination thereof.

The controller(s) 1036 may provide the signals for controlling one ormore components and/or systems of the vehicle 1000 in response to sensordata received from one or more sensors (e.g., sensor inputs). The sensordata may be received from, for example and without limitation, globalnavigation satellite systems sensor(s) 1058 (e.g., Global PositioningSystem sensor(s)), RADAR sensor(s) 1060, ultrasonic sensor(s) 1062,LIDAR sensor(s) 1064, inertial measurement unit (MU) sensor(s) 1066(e.g., accelerometer(s), gyroscope(s), magnetic compass(es),magnetometer(s), etc.), microphone(s) 1096, stereo camera(s) 1068,wide-view camera(s) 1070 (e.g., fisheye cameras), infrared camera(s)1072, surround camera(s) 1074 (e.g., 360 degree cameras), long-rangeand/or mid-range camera(s) 1098, speed sensor(s) 1044 (e.g., formeasuring the speed of the vehicle 1000), vibration sensor(s) 1042,steering sensor(s) 1040, brake sensor(s) (e.g., as part of the brakesensor system 1046), and/or other sensor types.

One or more of the controller(s) 1036 may receive inputs (e.g.,represented by input data) from an instrument cluster 1032 of thevehicle 1000 and provide outputs (e.g., represented by output data,display data, etc.) via a human-machine interface (HMI) display 1034, anaudible annunciator, a loudspeaker, and/or via other components of thevehicle 1000. The outputs may include information such as vehiclevelocity, speed, time, map data (e.g., the HD map 1022 of FIG. 0C),location data (e.g., the vehicle's 1000 location, such as on a map),direction, location of other vehicles (e.g., an occupancy grid),information about objects and status of objects as perceived by thecontroller(s) 1036, etc. For example, the HMI display 1034 may displayinformation about the presence of one or more objects (e.g., a streetsign, caution sign, traffic light changing, etc.), and/or informationabout driving maneuvers the vehicle has made, is making, or will make(e.g., changing lanes now, taking exit 34B in two miles, etc.).

The vehicle 1000 further includes a network interface 1024 which may useone or more wireless antenna(s) 1026 and/or modem(s) to communicate overone or more networks. For example, the network interface 1024 may becapable of communication over LTE, WCDMA, UMTS, GSM, CDMA2000, etc. Thewireless antenna(s) 1026 may also enable communication between objectsin the environment (e.g., vehicles, mobile devices, etc.), using localarea network(s), such as Bluetooth, Bluetooth LE, Z-Wave, ZigBee, etc.,and/or low power wide-area network(s) (LPWANs), such as LoRaWAN, SigFox,etc.

FIG. 10B is an example of camera locations and fields of view for theexample autonomous vehicle 1000 of FIG. 10A, in accordance with someembodiments of the present disclosure. The cameras and respective fieldsof view are one example embodiment and are not intended to be limiting.For example, additional and/or alternative cameras may be includedand/or the cameras may be located at different locations on the vehicle1000.

The camera types for the cameras may include, but are not limited to,digital cameras that may be adapted for use with the components and/orsystems of the vehicle 1000. The camera(s) may operate at automotivesafety integrity level (ASIL) B and/or at another ASIL. The camera typesmay be capable of any image capture rate, such as 60 frames per second(fps), 1020 fps, 240 fps, etc., depending on the embodiment. The camerasmay be capable of using rolling shutters, global shutters, another typeof shutter, or a combination thereof. In some examples, the color filterarray may include a red clear clear clear (RCCC) color filter array, ared clear clear blue (RCCB) color filter array, a red blue green clear(RBGC) color filter array, a Foveon X3 color filter array, a Bayersensors (RGGB) color filter array, a monochrome sensor color filterarray, and/or another type of color filter array. In some embodiments,clear pixel cameras, such as cameras with an RCCC, an RCCB, and/or anRBGC color filter array, may be used in an effort to increase lightsensitivity.

In some examples, one or more of the camera(s) may be used to performadvanced driver assistance systems (ADAS) functions (e.g., as part of aredundant or fail-safe design). For example, a Multi-Function MonoCamera may be installed to provide functions including lane departurewarning, traffic sign assist and intelligent headlamp control. One ormore of the camera(s) (e.g., all of the cameras) may record and provideimage data (e.g., video) simultaneously.

One or more of the cameras may be mounted in a mounting assembly, suchas a custom designed (3-D printed) assembly, in order to cut out straylight and reflections from within the car (e.g., reflections from thedashboard reflected in the windshield mirrors) which may interfere withthe camera's image data capture abilities. With reference to wing-mirrormounting assemblies, the wing-mirror assemblies may be custom 3-Dprinted so that the camera mounting plate matches the shape of thewing-mirror. In some examples, the camera(s) may be integrated into thewing-mirror. For side-view cameras, the camera(s) may also be integratedwithin the four pillars at each corner of the cabin.

Cameras with a field of view that include portions of the environment infront of the vehicle 1000 (e.g., front-facing cameras) may be used forsurround view, to help identify forward facing paths and obstacles, aswell aid in, with the help of one or more controllers 1036 and/orcontrol SoCs, providing information critical to generating an occupancygrid and/or determining the preferred vehicle paths. Front-facingcameras may be used to perform many of the same ADAS functions as LIDAR,including emergency braking, pedestrian detection, and collisionavoidance. Front-facing cameras may also be used for ADAS functions andsystems including Lane Departure Warnings (“LDW”), Autonomous CruiseControl (“ACC”), and/or other functions such as traffic signrecognition.

A variety of cameras may be used in a front-facing configuration,including, for example, a monocular camera platform that includes a CMOS(complementary metal oxide semiconductor) color imager. Another examplemay be a wide-view camera(s) 1070 that may be used to perceive objectscoming into view from the periphery (e.g., pedestrians, crossing trafficor bicycles). Although only one wide-view camera is illustrated in FIG.10B, there may any number of wide-view cameras 1070 on the vehicle 1000.In addition, long-range camera(s) 1098 (e.g., a long-view stereo camerapair) may be used for depth-based object detection, especially forobjects for which a neural network has not yet been trained. Thelong-range camera(s) 1098 may also be used for object detection andclassification, as well as basic object tracking.

One or more stereo cameras 1068 may also be included in a front-facingconfiguration. The stereo camera(s) 1068 may include an integratedcontrol unit comprising a scalable processing unit, which may provide aprogrammable logic (FPGA) and a multi-core micro-processor with anintegrated CAN or Ethernet interface on a single chip. Such a unit maybe used to generate a 3-D map of the vehicle's environment, including adistance estimate for all the points in the image. An alternative stereocamera(s) 1068 may include a compact stereo vision sensor(s) that mayinclude two camera lenses (one each on the left and right) and an imageprocessing chip that may measure the distance from the vehicle to thetarget object and use the generated information (e.g., metadata) toactivate the autonomous emergency braking and lane departure warningfunctions. Other types of stereo camera(s) 1068 may be used in additionto, or alternatively from, those described herein.

Cameras with a field of view that include portions of the environment tothe side of the vehicle 1000 (e.g., side-view cameras) may be used forsurround view, providing information used to create and update theoccupancy grid, as well as to generate side impact collision warnings.For example, surround camera(s) 1074 (e.g., four surround cameras 1074as illustrated in FIG. 10B) may be positioned to on the vehicle 1000.The surround camera(s) 1074 may include wide-view camera(s) 1070,fisheye camera(s), 360 degree camera(s), and/or the like. Four example,four fisheye cameras may be positioned on the vehicle's front, rear, andsides. In an alternative arrangement, the vehicle may use three surroundcamera(s) 1074 (e.g., left, right, and rear), and may leverage one ormore other camera(s) (e.g., a forward-facing camera) as a fourthsurround view camera.

Cameras with a field of view that include portions of the environment tothe rear of the vehicle 1000 (e.g., rear-view cameras) may be used forpark assistance, surround view, rear collision warnings, and creatingand updating the occupancy grid. A wide variety of cameras may be usedincluding, but not limited to, cameras that are also suitable as afront-facing camera(s) (e.g., long-range and/or mid-range camera(s)1098, stereo camera(s) 1068), infrared camera(s) 1072, etc.), asdescribed herein.

FIG. 10C is a block diagram of an example system architecture for theexample autonomous vehicle 1000 of FIG. 10A, in accordance with someembodiments of the present disclosure. It should be understood that thisand other arrangements described herein are set forth only as examples.Other arrangements and elements (e.g., machines, interfaces, functions,orders, groupings of functions, etc.) may be used in addition to orinstead of those shown, and some elements may be omitted altogether.Further, many of the elements described herein are functional entitiesthat may be implemented as discrete or distributed components or inconjunction with other components, and in any suitable combination andlocation. Various functions described herein as being performed byentities may be carried out by hardware, firmware, and/or software. Forinstance, various functions may be carried out by a processor executinginstructions stored in memory.

Each of the components, features, and systems of the vehicle 1000 inFIG. 10C are illustrated as being connected via bus 1002. The bus 1002may include a Controller Area Network (CAN) data interface(alternatively referred to herein as a “CAN bus”). A CAN may be anetwork inside the vehicle 1000 used to aid in control of variousfeatures and functionality of the vehicle 1000, such as actuation ofbrakes, acceleration, braking, steering, windshield wipers, etc. A CANbus may be configured to have dozens or even hundreds of nodes, eachwith its own unique identifier (e.g., a CAN ID). The CAN bus may be readto find steering wheel angle, ground speed, engine revolutions perminute (RPMs), button positions, and/or other vehicle status indicators.The CAN bus may be ASIL B compliant.

Although the bus 1002 is described herein as being a CAN bus, this isnot intended to be limiting. For example, in addition to, oralternatively from, the CAN bus, FlexRay and/or Ethernet may be used.Additionally, although a single line is used to represent the bus 1002,this is not intended to be limiting. For example, there may be anynumber of busses 1002, which may include one or more CAN busses, one ormore FlexRay busses, one or more Ethernet busses, and/or one or moreother types of busses using a different protocol. In some examples, twoor more busses 1002 may be used to perform different functions, and/ormay be used for redundancy. For example, a first bus 1002 may be usedfor collision avoidance functionality and a second bus 1002 may be usedfor actuation control. In any example, each bus 1002 may communicatewith any of the components of the vehicle 1000, and two or more busses1002 may communicate with the same components. In some examples, eachSoC 1004, each controller 1036, and/or each computer within the vehiclemay have access to the same input data (e.g., inputs from sensors of thevehicle 1000), and may be connected to a common bus, such the CAN bus.

The vehicle 1000 may include one or more controller(s) 1036, such asthose described herein with respect to FIG. 10A. The controller(s) 1036may be used for a variety of functions. The controller(s) 1036 may becoupled to any of the various other components and systems of thevehicle 1000, and may be used for control of the vehicle 1000,artificial intelligence of the vehicle 1000, infotainment for thevehicle 1000, and/or the like.

The vehicle 1000 may include a system(s) on a chip (SoC) 1004. The SoC1004 may include CPU(s) 1006, GPU(s) 1008, processor(s) 1010, cache(s)1012, accelerator(s) 1014, data store(s) 1016, and/or other componentsand features not illustrated. The SoC(s) 1004 may be used to control thevehicle 1000 in a variety of platforms and systems. For example, theSoC(s) 1004 may be combined in a system (e.g., the system of the vehicle1000) with an HD map 1022 which may obtain map refreshes and/or updatesvia a network interface 1024 from one or more servers (e.g., server(s)1078 of FIG. 10D).

The CPU(s) 1006 may include a CPU cluster or CPU complex (alternativelyreferred to herein as a “CCPLEX”). The CPU(s) 1006 may include multiplecores and/or L2 caches. For example, in some embodiments, the CPU(s)1006 may include eight cores in a coherent multi-processorconfiguration. In some embodiments, the CPU(s) 1006 may include fourdual-core clusters where each cluster has a dedicated L2 cache (e.g., a2 MB L2 cache). The CPU(s) 1006 (e.g., the CCPLEX) may be configured tosupport simultaneous cluster operation enabling any combination of theclusters of the CPU(s) 1006 to be active at any given time.

The CPU(s) 1006 may implement power management capabilities that includeone or more of the following features: individual hardware blocks may beclock-gated automatically when idle to save dynamic power; each coreclock may be gated when the core is not actively executing instructionsdue to execution of WFI/WFE instructions; each core may be independentlypower-gated; each core cluster may be independently clock-gated when allcores are clock-gated or power-gated; and/or each core cluster may beindependently power-gated when all cores are power-gated. The CPU(s)1006 may further implement an enhanced algorithm for managing powerstates, where allowed power states and expected wakeup times arespecified, and the hardware/microcode determines the best power state toenter for the core, cluster, and CCPLEX. The processing cores maysupport simplified power state entry sequences in software with the workoffloaded to microcode.

The GPU(s) 1008 may include an integrated GPU (alternatively referred toherein as an “iGPU”). The GPU(s) 1008 may be programmable and may beefficient for parallel workloads. The GPU(s) 1008, in some examples, mayuse an enhanced tensor instruction set. The GPU(s) 1008 may include oneor more streaming microprocessors, where each streaming microprocessormay include an L1 cache (e.g., an L1 cache with at least 96 KB storagecapacity), and two or more of the streaming microprocessors may share anL2 cache (e.g., an L2 cache with a 512 KB storage capacity). In someembodiments, the GPU(s) 1008 may include at least eight streamingmicroprocessors. The GPU(s) 1008 may use compute application programminginterface(s) (API(s)). In addition, the GPU(s) 1008 may use one or moreparallel computing platforms and/or programming models (e.g., NVIDIA'sCUDA).

The GPU(s) 1008 may be power-optimized for best performance inautomotive and embedded use cases. For example, the GPU(s) 1008 may befabricated on a Fin field-effect transistor (FinFET). However, this isnot intended to be limiting and the GPU(s) 1008 may be fabricated usingother semiconductor manufacturing processes. Each streamingmicroprocessor may incorporate a number of mixed-precision processingcores partitioned into multiple blocks. For example, and withoutlimitation, 64 PF32 cores and 32 PF64 cores may be partitioned into fourprocessing blocks. In such an example, each processing block may beallocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, twomixed-precision NVIDIA TENSOR COREs for deep learning matrix arithmetic,an L0 instruction cache, a warp scheduler, a dispatch unit, and/or a 64KB register file. In addition, the streaming microprocessors may includeindependent parallel integer and floating-point data paths to providefor efficient execution of workloads with a mix of computation andaddressing calculations. The streaming microprocessors may includeindependent thread scheduling capability to enable finer-grainsynchronization and cooperation between parallel threads. The streamingmicroprocessors may include a combined L1 data cache and shared memoryunit in order to improve performance while simplifying programming.

The GPU(s) 1008 may include a high bandwidth memory (HBM) and/or a 16 GBHBM2 memory subsystem to provide, in some examples, about 900 GB/secondpeak memory bandwidth. In some examples, in addition to, oralternatively from, the HBM memory, a synchronous graphics random-accessmemory (SGRAM) may be used, such as a graphics double data rate typefive synchronous random-access memory (GDDR5).

The GPU(s) 1008 may include unified memory technology including accesscounters to allow for more accurate migration of memory pages to theprocessor that accesses them most frequently, thereby improvingefficiency for memory ranges shared between processors. In someexamples, address translation services (ATS) support may be used toallow the GPU(s) 1008 to access the CPU(s) 1006 page tables directly. Insuch examples, when the GPU(s) 1008 memory management unit (MMU)experiences a miss, an address translation request may be transmitted tothe CPU(s) 1006. In response, the CPU(s) 1006 may look in its pagetables for the virtual-to-physical mapping for the address and transmitsthe translation back to the GPU(s) 1008. As such, unified memorytechnology may allow a single unified virtual address space for memoryof both the CPU(s) 1006 and the GPU(s) 1008, thereby simplifying theGPU(s) 1008 programming and porting of applications to the GPU(s) 1008.

In addition, the GPU(s) 1008 may include an access counter that may keeptrack of the frequency of access of the GPU(s) 1008 to memory of otherprocessors. The access counter may help ensure that memory pages aremoved to the physical memory of the processor that is accessing thepages most frequently.

The SoC(s) 1004 may include any number of cache(s) 1012, including thosedescribed herein. For example, the cache(s) 1012 may include an L3 cachethat is available to both the CPU(s) 1006 and the GPU(s) 1008 (e.g.,that is connected both the CPU(s) 1006 and the GPU(s) 1008). Thecache(s) 1012 may include a write-back cache that may keep track ofstates of lines, such as by using a cache coherence protocol (e.g., MEI,MESI, MSI, etc.). The L3 cache may include 4 MB or more, depending onthe embodiment, although smaller cache sizes may be used.

The SoC(s) 1004 may include one or more accelerators 1014 (e.g.,hardware accelerators, software accelerators, or a combination thereof).For example, the SoC(s) 1004 may include a hardware acceleration clusterthat may include optimized hardware accelerators and/or large on-chipmemory. The large on-chip memory (e.g., 4 MB of SRAM), may enable thehardware acceleration cluster to accelerate neural networks and othercalculations. The hardware acceleration cluster may be used tocomplement the GPU(s) 1008 and to off-load some of the tasks of theGPU(s) 1008 (e.g., to free up more cycles of the GPU(s) 1008 forperforming other tasks). As an example, the accelerator(s) 1014 may beused for targeted workloads (e.g., perception, convolutional neuralnetworks (CNNs), etc.) that are stable enough to be amenable toacceleration. The term “CNN,” as used herein, may include all types ofCNNs, including region-based or regional convolutional neural networks(RCNNs) and Fast RCNNs (e.g., as used for object detection).

The accelerator(s) 1014 (e.g., the hardware acceleration cluster) mayinclude a deep learning accelerator(s) (DLA). The DLA(s) may include oneor more Tensor processing units (TPUs) that may be configured to providean additional ten trillion operations per second for deep learningapplications and inferencing. The TPUs may be accelerators configuredto, and optimized for, performing image processing functions (e.g., forCNNs, RCNNs, etc.). The DLA(s) may further be optimized for a specificset of neural network types and floating point operations, as well asinferencing. The design of the DLA(s) may provide more performance permillimeter than a general-purpose GPU, and vastly exceeds theperformance of a CPU. The TPU(s) may perform several functions,including a single-instance convolution function, supporting, forexample, INT8, INT16, and FP16 data types for both features and weights,as well as post-processor functions.

The DLA(s) may quickly and efficiently execute neural networks,especially CNNs, on processed or unprocessed data for any of a varietyof functions, including, for example and without limitation: a CNN forobject identification and detection using data from camera sensors; aCNN for distance estimation using data from camera sensors; a CNN foremergency vehicle detection and identification and detection using datafrom microphones; a CNN for facial recognition and vehicle owneridentification using data from camera sensors; and/or a CNN for securityand/or safety related events.

The DLA(s) may perform any function of the GPU(s) 1008, and by using aninference accelerator, for example, a designer may target either theDLA(s) or the GPU(s) 1008 for any function. For example, the designermay focus processing of CNNs and floating point operations on the DLA(s)and leave other functions to the GPU(s) 1008 and/or other accelerator(s)1014.

The accelerator(s) 1014 (e.g., the hardware acceleration cluster) mayinclude a programmable vision accelerator(s) (PVA), which mayalternatively be referred to herein as a computer vision accelerator.The PVA(s) may be designed and configured to accelerate computer visionalgorithms for the advanced driver assistance systems (ADAS), autonomousdriving, and/or augmented reality (AR) and/or virtual reality (VR)applications. The PVA(s) may provide a balance between performance andflexibility. For example, each PVA(s) may include, for example andwithout limitation, any number of reduced instruction set computer(RISC) cores, direct memory access (DMA), and/or any number of vectorprocessors.

The RISC cores may interact with image sensors (e.g., the image sensorsof any of the cameras described herein), image signal processor(s),and/or the like. Each of the RISC cores may include any amount ofmemory. The RISC cores may use any of a number of protocols, dependingon the embodiment. In some examples, the RISC cores may execute areal-time operating system (RTOS). The RISC cores may be implementedusing one or more integrated circuit devices, application specificintegrated circuits (ASICs), and/or memory devices. For example, theRISC cores may include an instruction cache and/or a tightly coupledRAM.

The DMA may enable components of the PVA(s) to access the system memoryindependently of the CPU(s) 1006. The DMA may support any number offeatures used to provide optimization to the PVA including, but notlimited to, supporting multi-dimensional addressing and/or circularaddressing. In some examples, the DMA may support up to six or moredimensions of addressing, which may include block width, block height,block depth, horizontal block stepping, vertical block stepping, and/ordepth stepping.

The vector processors may be programmable processors that may bedesigned to efficiently and flexibly execute programming for computervision algorithms and provide signal processing capabilities. In someexamples, the PVA may include a PVA core and two vector processingsubsystem partitions. The PVA core may include a processor subsystem,DMA engine(s) (e.g., two DMA engines), and/or other peripherals. Thevector processing subsystem may operate as the primary processing engineof the PVA, and may include a vector processing unit (VPU), aninstruction cache, and/or vector memory (e.g., VMEM). A VPU core mayinclude a digital signal processor such as, for example, a singleinstruction, multiple data (SIMD), very long instruction word (VLIW)digital signal processor. The combination of the SIMD and VLIW mayenhance throughput and speed.

Each of the vector processors may include an instruction cache and maybe coupled to dedicated memory. As a result, in some examples, each ofthe vector processors may be configured to execute independently of theother vector processors. In other examples, the vector processors thatare included in a particular PVA may be configured to employ dataparallelism. For example, in some embodiments, the plurality of vectorprocessors included in a single PVA may execute the same computer visionalgorithm, but on different regions of an image. In other examples, thevector processors included in a particular PVA may simultaneouslyexecute different computer vision algorithms, on the same image, or evenexecute different algorithms on sequential images or portions of animage. Among other things, any number of PVAs may be included in thehardware acceleration cluster and any number of vector processors may beincluded in each of the PVAs. In addition, the PVA(s) may includeadditional error correcting code (ECC) memory, to enhance overall systemsafety.

The accelerator(s) 1014 (e.g., the hardware acceleration cluster) mayinclude a computer vision network on-chip and SRAM, for providing ahigh-bandwidth, low latency SRAM for the accelerator(s) 1014. In someexamples, the on-chip memory may include at least 4 MB SRAM, consistingof, for example and without limitation, eight field-configurable memoryblocks, that may be accessible by both the PVA and the DLA. Each pair ofmemory blocks may include an advanced peripheral bus (APB) interface,configuration circuitry, a controller, and a multiplexer. Any type ofmemory may be used. The PVA and DLA may access the memory via a backbonethat provides the PVA and DLA with high-speed access to memory. Thebackbone may include a computer vision network on-chip thatinterconnects the PVA and the DLA to the memory (e.g., using the APB).

The computer vision network on-chip may include an interface thatdetermines, before transmission of any control signal/address/data, thatboth the PVA and the DLA provide ready and valid signals. Such aninterface may provide for separate phases and separate channels fortransmitting control signals/addresses/data, as well as burst-typecommunications for continuous data transfer. This type of interface maycomply with ISO 26262 or IEC 61508 standards, although other standardsand protocols may be used.

In some examples, the SoC(s) 1004 may include a real-time ray-tracinghardware accelerator, such as described in U.S. patent application Ser.No. 16/101,232, filed on Aug. 10, 2018. The real-time ray-tracinghardware accelerator may be used to quickly and efficiently determinethe positions and extents of objects (e.g., within a world model), togenerate real0time visualization simulations, for RADAR signalinterpretation, for sound propagation synthesis and/or analysis, forsimulation of SONAR systems, for general wave propagation simulation,for comparison to LIDAR data for purposes of localization and/or otherfunctions, and/or for other uses.

The accelerator(s) 1014 (e.g., the hardware accelerator cluster) have awide array of uses for autonomous driving. The PVA may be a programmablevision accelerator that may be used for key processing stages in ADASand autonomous vehicles. The PVA's capabilities are a good match foralgorithmic domains needing predictable processing, at low power and lowlatency. In other words, the PVA performs well on semi-dense or denseregular computation, even on small data sets, which need predictablerun-times with low latency and low power. Thus, in the context ofplatforms for autonomous vehicles, the PVAs are designed to run classiccomputer vision algorithms, as they are efficient at object detectionand operating on integer math.

For example, according to one embodiment of the technology, the PVA isused to perform computer stereo vision. A semi-global matching-basedalgorithm may be used in some examples, although this is not intended tobe limiting. Many applications for Level 3-5 autonomous driving requiremotion estimation/stereo matching on-the-fly (e.g., structure frommotion, pedestrian recognition, lane detection, etc.). The PVA mayperform computer stereo vision function on inputs from two monocularcameras.

In some examples, the PVA may be used to perform dense optical flow.According to process raw RADAR data (e.g., using a 4D Fast FourierTransform) to provide Processed RADAR. In other examples, the PVA isused for time of flight depth processing, by processing raw time offlight data to provide processed time of flight data, for example.

The DLA may be used to run any type of network to enhance control anddriving safety, including for example, a neural network that outputs ameasure of confidence for each object detection. Such a confidence valuemay be interpreted as a probability, or as providing a relative “weight”of each detection compared to other detections. This confidence valueenables the system to make further decisions regarding which detectionsshould be considered as true positive detections rather than falsepositive detections. For example, the system may set a threshold valuefor the confidence and consider only the detections exceeding thethreshold value as true positive detections. In an automatic emergencybraking (AEB) system, false positive detections would cause the vehicleto automatically perform emergency braking, which is obviouslyundesirable. Therefore, only the most confident detections should beconsidered as triggers for AEB. The DLA may run a neural network forregressing the confidence value. The neural network may take as itsinput at least some subset of parameters, such as bounding boxdimensions, ground plane estimate obtained (e.g. from anothersubsystem), inertial measurement unit (IMU) sensor 1066 output thatcorrelates with the vehicle 1000 orientation, distance, 3D locationestimates of the object obtained from the neural network and/or othersensors (e.g., LIDAR sensor(s) 1064 or RADAR sensor(s) 1060), amongothers.

The SoC(s) 1004 may include data store(s) 1016 (e.g., memory). The datastore(s) 1016 may be on-chip memory of the SoC(s) 1004, which may storeneural networks to be executed on the GPU and/or the DLA. In someexamples, the data store(s) 1016 may be large enough in capacity tostore multiple instances of neural networks for redundancy and safety.The data store(s) 1012 may comprise L2 or L3 cache(s) 1012. Reference tothe data store(s) 1016 may include reference to the memory associatedwith the PVA, DLA, and/or other accelerator(s) 1014, as describedherein.

The SoC(s) 1004 may include one or more processor(s) 1010 (e.g.,embedded processors). The processor(s) 1010 may include a boot and powermanagement processor that may be a dedicated processor and subsystem tohandle boot power and management functions and related securityenforcement. The boot and power management processor may be a part ofthe SoC(s) 1004 boot sequence and may provide runtime power managementservices. The boot power and management processor may provide clock andvoltage programming, assistance in system low power state transitions,management of SoC(s) 1004 thermals and temperature sensors, and/ormanagement of the SoC(s) 1004 power states. Each temperature sensor maybe implemented as a ring-oscillator whose output frequency isproportional to temperature, and the SoC(s) 1004 may use thering-oscillators to detect temperatures of the CPU(s) 1006, GPU(s) 1008,and/or accelerator(s) 1014. If temperatures are determined to exceed athreshold, the boot and power management processor may enter atemperature fault routine and put the SoC(s) 1004 into a lower powerstate and/or put the vehicle 1000 into a chauffeur to safe stop mode(e.g., bring the vehicle 1000 to a safe stop).

The processor(s) 1010 may further include a set of embedded processorsthat may serve as an audio processing engine. The audio processingengine may be an audio subsystem that enables full hardware support formulti-channel audio over multiple interfaces, and a broad and flexiblerange of audio I/O interfaces. In some examples, the audio processingengine is a dedicated processor core with a digital signal processorwith dedicated RAM.

The processor(s) 1010 may further include an always on processor enginethat may provide necessary hardware features to support low power sensormanagement and wake use cases. The always on processor engine mayinclude a processor core, a tightly coupled RAM, supporting peripherals(e.g., timers and interrupt controllers), various I/O controllerperipherals, and routing logic.

The processor(s) 1010 may further include a safety cluster engine thatincludes a dedicated processor subsystem to handle safety management forautomotive applications. The safety cluster engine may include two ormore processor cores, a tightly coupled RAM, support peripherals (e.g.,timers, an interrupt controller, etc.), and/or routing logic. In asafety mode, the two or more cores may operate in a lockstep mode andfunction as a single core with comparison logic to detect anydifferences between their operations.

The processor(s) 1010 may further include a real-time camera engine thatmay include a dedicated processor subsystem for handling real-timecamera management.

The processor(s) 1010 may further include a high-dynamic range signalprocessor that may include an image signal processor that is a hardwareengine that is part of the camera processing pipeline.

The processor(s) 1010 may include a video image compositor that may be aprocessing block (e.g., implemented on a microprocessor) that implementsvideo post-processing functions needed by a video playback applicationto produce the final image for the player window. The video imagecompositor may perform lens distortion correction on wide-view camera(s)1070, surround camera(s) 1074, and/or on in-cabin monitoring camerasensors. In-cabin monitoring camera sensor is preferably monitored by aneural network running on another instance of the Advanced SoC,configured to identify in cabin events and respond accordingly. Anin-cabin system may perform lip reading to activate cellular service andplace a phone call, dictate emails, change the vehicle's destination,activate or change the vehicle's infotainment system and settings, orprovide voice-activated web surfing. Certain functions are available tothe driver only when the vehicle is operating in an autonomous mode, andare disabled otherwise.

The video image compositor may include enhanced temporal noise reductionfor both spatial and temporal noise reduction. For example, where motionoccurs in a video, the noise reduction weights spatial informationappropriately, decreasing the weight of information provided by adjacentframes. Where an image or portion of an image does not include motion,the temporal noise reduction performed by the video image compositor mayuse information from the previous image to reduce noise in the currentimage.

The video image compositor may also be configured to perform stereorectification on input stereo lens frames. The video image compositormay further be used for user interface composition when the operatingsystem desktop is in use, and the GPU(s) 1008 is not required tocontinuously render new surfaces. Even when the GPU(s) 1008 is poweredon and active doing 3D rendering, the video image compositor may be usedto offload the GPU(s) 1008 to improve performance and responsiveness.

The SoC(s) 1004 may further include a mobile industry processorinterface (MIPI) camera serial interface for receiving video and inputfrom cameras, a high-speed interface, and/or a video input block thatmay be used for camera and related pixel input functions. The SoC(s)1004 may further include an input/output controller(s) that may becontrolled by software and may be used for receiving I/O signals thatare uncommitted to a specific role.

The SoC(s) 1004 may further include a broad range of peripheralinterfaces to enable communication with peripherals, audio codecs, powermanagement, and/or other devices. The SoC(s) 1004 may be used to processdata from cameras (e.g., connected over Gigabit Multimedia Serial Linkand Ethernet), sensors (e.g., LIDAR sensor(s) 1064, RADAR sensor(s)1060, etc. that may be connected over Ethernet), data from bus 1002(e.g., speed of vehicle 1000, steering wheel position, etc.), data fromGNSS sensor(s) 1058 (e.g., connected over Ethernet or CAN bus). TheSoC(s) 1004 may further include dedicated high-performance mass storagecontrollers that may include their own DMA engines, and that may be usedto free the CPU(s) 1006 from routine data management tasks.

The SoC(s) 1004 may be an end-to-end platform with a flexiblearchitecture that spans automation levels 3-5, thereby providing acomprehensive functional safety architecture that leverages and makesefficient use of computer vision and ADAS techniques for diversity andredundancy, provides a platform for a flexible, reliable drivingsoftware stack, along with deep learning tools. The SoC(s) 1004 may befaster, more reliable, and even more energy-efficient andspace-efficient than conventional systems. For example, theaccelerator(s) 1014, when combined with the CPU(s) 1006, the GPU(s)1008, and the data store(s) 1016, may provide for a fast, efficientplatform for level 3-5 autonomous vehicles.

The technology thus provides capabilities and functionality that cannotbe achieved by conventional systems. For example, computer visionalgorithms may be executed on CPUs, which may be configured usinghigh-level programming language, such as the C programming language, toexecute a wide variety of processing algorithms across a wide variety ofvisual data. However, CPUs are oftentimes unable to meet the performancerequirements of many computer vision applications, such as those relatedto execution time and power consumption, for example. In particular,many CPUs are unable to execute complex object detection algorithms inreal-time, which is a requirement of in-vehicle ADAS applications, and arequirement for practical Level 3-5 autonomous vehicles.

In contrast to conventional systems, by providing a CPU complex, GPUcomplex, and a hardware acceleration cluster, the technology describedherein allows for multiple neural networks to be performedsimultaneously and/or sequentially, and for the results to be combinedtogether to enable Level 3-5 autonomous driving functionality. Forexample, a CNN executing on the DLA or dGPU (e.g., the GPU(s) 1020) mayinclude a text and word recognition, allowing the supercomputer to readand understand traffic signs, including signs for which the neuralnetwork has not been specifically trained. The DLA may further include aneural network that is able to identify, interpret, and providessemantic understanding of the sign, and to pass that semanticunderstanding to the path planning modules running on the CPU Complex.

As another example, multiple neural networks may be run simultaneously,as is required for Level 3, 4, or 5 driving. For example, a warning signconsisting of “Caution: flashing lights indicate icy conditions,” alongwith an electric light, may be independently or collectively interpretedby several neural networks. The sign itself may be identified as atraffic sign by a first deployed neural network (e.g., a neural networkthat has been trained), the text “Flashing lights indicate icyconditions” may be interpreted by a second deployed neural network,which informs the vehicle's path planning software (preferably executingon the CPU Complex) that when flashing lights are detected, icyconditions exist. The flashing light may be identified by operating athird deployed neural network over multiple frames, informing thevehicle's path-planning software of the presence (or absence) offlashing lights. All three neural networks may run simultaneously, suchas within the DLA and/or on the GPU(s) 1008.

In some examples, a CNN for facial recognition and vehicle owneridentification may use data from camera sensors to identify the presenceof an authorized driver and/or owner of the vehicle 1000. The always onsensor processing engine may be used to unlock the vehicle when theowner approaches the driver door and turn on the lights, and, insecurity mode, to disable the vehicle when the owner leaves the vehicle.In this way, the SoC(s) 1004 provide for security against theft and/orcarjacking.

In another example, a CNN for emergency vehicle detection andidentification may use data from microphones 1096 to detect and identifyemergency vehicle sirens. In contrast to conventional systems, that usegeneral classifiers to detect sirens and manually extract features, theSoC(s) 1004 use the CNN for classifying environmental and urban sounds,as well as classifying visual data. In a preferred embodiment, the CNNrunning on the DLA is trained to identify the relative closing speed ofthe emergency vehicle (e.g., by using the Doppler Effect). The CNN mayalso be trained to identify emergency vehicles specific to the localarea in which the vehicle is operating, as identified by GNSS sensor(s)1058. Thus, for example, when operating in Europe the CNN will seek todetect European sirens, and when in the United States the CNN will seekto identify only North American sirens. Once an emergency vehicle isdetected, a control program may be used to execute an emergency vehiclesafety routine, slowing the vehicle, pulling over to the side of theroad, parking the vehicle, and/or idling the vehicle, with theassistance of ultrasonic sensors 1062, until the emergency vehicle(s)passes.

The vehicle may include a CPU(s) 1018 (e.g., discrete CPU(s), ordCPU(s)), that may be coupled to the SoC(s) 1004 via a high-speedinterconnect (e.g., PCIe). The CPU(s) 1018 may include an X86 processor,for example. The CPU(s) 1018 may be used to perform any of a variety offunctions, including arbitrating potentially inconsistent resultsbetween ADAS sensors and the SoC(s) 1004, and/or monitoring the statusand health of the controller(s) 1036 and/or infotainment SoC 1030, forexample.

The vehicle 1000 may include a GPU(s) 1020 (e.g., discrete GPU(s), ordGPU(s)), that may be coupled to the SoC(s) 1004 via a high-speedinterconnect (e.g., NVIDIA's NVLINK). The GPU(s) 1020 may provideadditional artificial intelligence functionality, such as by executingredundant and/or different neural networks, and may be used to trainand/or update neural networks based on input (e.g., sensor data) fromsensors of the vehicle 1000.

The vehicle 1000 may further include the network interface 1024 whichmay include one or more wireless antennas 1026 (e.g., one or morewireless antennas for different communication protocols, such as acellular antenna, a Bluetooth antenna, etc.). The network interface 1024may be used to enable wireless connectivity over the Internet with thecloud (e.g., with the server(s) 1078 and/or other network devices), withother vehicles, and/or with computing devices (e.g., client devices ofpassengers). To communicate with other vehicles, a direct link may beestablished between the two vehicles and/or an indirect link may beestablished (e.g., across networks and over the Internet). Direct linksmay be provided using a vehicle-to-vehicle communication link. Thevehicle-to-vehicle communication link may provide the vehicle 1000information about vehicles in proximity to the vehicle 1000 (e.g.,vehicles in front of, on the side of, and/or behind the vehicle 1000).This functionality may be part of a cooperative adaptive cruise controlfunctionality of the vehicle 1000.

The network interface 1024 may include a SoC that provides modulationand demodulation functionality and enables the controller(s) 1036 tocommunicate over wireless networks. The network interface 1024 mayinclude a radio frequency front-end for up-conversion from baseband toradio frequency, and down conversion from radio frequency to baseband.The frequency conversions may be performed through well-known processes,and/or may be performed using super-heterodyne processes. In someexamples, the radio frequency front end functionality may be provided bya separate chip. The network interface may include wirelessfunctionality for communicating over LTE, WCDMA, UMTS, GSM, CDMA2000,Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWAN, and/or otherwireless protocols.

The vehicle 1000 may further include data store(s) 1028 which mayinclude off-chip (e.g., off the SoC(s) 1004) storage. The data store(s)1028 may include one or more storage elements including RAM, SRAM, DRAM,VRAM, Flash, hard disks, and/or other components and/or devices that maystore at least one bit of data.

The vehicle 1000 may further include GNSS sensor(s) 1058. The GNSSsensor(s) 1058 (e.g., GPS and/or assisted GPS sensors), to assist inmapping, perception, occupancy grid generation, and/or path planningfunctions. Any number of GNSS sensor(s) 1058 may be used, including, forexample and without limitation, a GPS using a USB connector with anEthernet to Serial (RS-232) bridge.

The vehicle 1000 may further include RADAR sensor(s) 1060. The RADARsensor(s) 1060 may be used by the vehicle 1000 for long-range vehicledetection, even in darkness and/or severe weather conditions. RADARfunctional safety levels may be ASIL B. The RADAR sensor(s) 1060 may usethe CAN and/or the bus 1002 (e.g., to transmit data generated by theRADAR sensor(s) 1060) for control and to access object tracking data,with access to Ethernet to access raw data in some examples. A widevariety of RADAR sensor types may be used. For example, and withoutlimitation, the RADAR sensor(s) 1060 may be suitable for front, rear,and side RADAR use. In some example, Pulse Doppler RADAR sensor(s) areused.

The RADAR sensor(s) 1060 may include different configurations, such aslong range with narrow field of view, short range with wide field ofview, short range side coverage, etc. In some examples, long-range RADARmay be used for adaptive cruise control functionality. The long-rangeRADAR systems may provide a broad field of view realized by two or moreindependent scans, such as within a 250 m range. The RADAR sensor(s)1060 may help in distinguishing between static and moving objects, andmay be used by ADAS systems for emergency brake assist and forwardcollision warning. Long-range RADAR sensors may include monostaticmultimodal RADAR with multiple (e.g., six or more) fixed RADAR antennaeand a high-speed CAN and FlexRay interface. In an example with sixantennae, the central four antennae may create a focused beam pattern,designed to record the vehicle's 1000 surroundings at higher speeds withminimal interference from traffic in adjacent lanes. The other twoantennae may expand the field of view, making it possible to quicklydetect vehicles entering or leaving the vehicle's 1000 lane.

Mid-range RADAR systems may include, as an example, a range of up to1060 m (front) or 80 m (rear), and a field of view of up to 42 degrees(front) or 1050 degrees (rear). Short-range RADAR systems may include,without limitation, RADAR sensors designed to be installed at both endsof the rear bumper. When installed at both ends of the rear bumper, sucha RADAR sensor systems may create two beams that constantly monitor theblind spot in the rear and next to the vehicle.

Short-range RADAR systems may be used in an ADAS system for blind spotdetection and/or lane change assist.

The vehicle 1000 may further include ultrasonic sensor(s) 1062. Theultrasonic sensor(s) 1062, which may be positioned at the front, back,and/or the sides of the vehicle 1000, may be used for park assist and/orto create and update an occupancy grid. A wide variety of ultrasonicsensor(s) 1062 may be used, and different ultrasonic sensor(s) 1062 maybe used for different ranges of detection (e.g., 2.5 m, 4 m). Theultrasonic sensor(s) 1062 may operate at functional safety levels ofASIL B.

The vehicle 1000 may include LIDAR sensor(s) 1064. The LIDAR sensor(s)1064 may be used for object and pedestrian detection, emergency braking,collision avoidance, and/or other functions. The LIDAR sensor(s) 1064may be functional safety level ASIL B. In some examples, the vehicle1000 may include multiple LIDAR sensors 1064 (e.g., two, four, six,etc.) that may use Ethernet (e.g., to provide data to a Gigabit Ethernetswitch).

In some examples, the LIDAR sensor(s) 1064 may be capable of providing alist of objects and their distances for a 360-degree field of view.Commercially available LIDAR sensor(s) 1064 may have an advertised rangeof approximately 1000 m, with an accuracy of 2 cm-3 cm, and with supportfor a 1000 Mbps Ethernet connection, for example. In some examples, oneor more non-protruding LIDAR sensors 1064 may be used. In such examples,the LIDAR sensor(s) 1064 may be implemented as a small device that maybe embedded into the front, rear, sides, and/or corners of the vehicle1000. The LIDAR sensor(s) 1064, in such examples, may provide up to a1020-degree horizontal and 35-degree vertical field-of-view, with a 200m range even for low-reflectivity objects. Front-mounted LIDAR sensor(s)1064 may be configured for a horizontal field of view between 45 degreesand 135 degrees.

In some examples, LIDAR technologies, such as 3D flash LIDAR, may alsobe used. 3D Flash LIDAR uses a flash of a laser as a transmissionsource, to illuminate vehicle surroundings up to approximately 200 m. Aflash LIDAR unit includes a receptor, which records the laser pulsetransit time and the reflected light on each pixel, which in turncorresponds to the range from the vehicle to the objects. Flash LIDARmay allow for highly accurate and distortion-free images of thesurroundings to be generated with every laser flash. In some examples,four flash LIDAR sensors may be deployed, one at each side of thevehicle 1000. Available 3D flash LIDAR systems include a solid-state 3Dstaring array LIDAR camera with no moving parts other than a fan (e.g.,a non-scanning LIDAR device). The flash LIDAR device may use a 5nanosecond class I (eye-safe) laser pulse per frame and may capture thereflected laser light in the form of 3D range point clouds andco-registered intensity data. By using flash LIDAR, and because flashLIDAR is a solid-state device with no moving parts, the LIDAR sensor(s)1064 may be less susceptible to motion blur, vibration, and/or shock.

The vehicle may further include IMU sensor(s) 1066. The IMU sensor(s)1066 may be located at a center of the rear axle of the vehicle 1000, insome examples. The IMU sensor(s) 1066 may include, for example andwithout limitation, an accelerometer(s), a magnetometer(s), agyroscope(s), a magnetic compass(es), and/or other sensor types. In someexamples, such as in six-axis applications, the IMU sensor(s) 1066 mayinclude accelerometers and gyroscopes, while in nine-axis applications,the IMU sensor(s) 1066 may include accelerometers, gyroscopes, andmagnetometers.

In some embodiments, the IMU sensor(s) 1066 may be implemented as aminiature, high performance GPS-Aided Inertial Navigation System(GPS/INS) that combines micro-electro-mechanical systems (MEMS) inertialsensors, a high-sensitivity GPS receiver, and advanced Kalman filteringalgorithms to provide estimates of position, velocity, and attitude. Assuch, in some examples, the IMU sensor(s) 1066 may enable the vehicle1000 to estimate heading without requiring input from a magnetic sensorby directly observing and correlating the changes in velocity from GPSto the IMU sensor(s) 1066. In some examples, the IMU sensor(s) 1066 andthe GNSS sensor(s) 1058 may be combined in a single integrated unit.

The vehicle may include microphone(s) 1096 placed in and/or around thevehicle 1000. The microphone(s) 1096 may be used for emergency vehicledetection and identification, among other things.

The vehicle may further include any number of camera types, includingstereo camera(s) 1068, wide-view camera(s) 1070, infrared camera(s)1072, surround camera(s) 1074, long-range and/or mid-range camera(s)1098, and/or other camera types. The cameras may be used to captureimage data around an entire periphery of the vehicle 1000. The types ofcameras used depends on the embodiments and requirements for the vehicle1000, and any combination of camera types may be used to provide thenecessary coverage around the vehicle 1000. In addition, the number ofcameras may differ depending on the embodiment. For example, the vehiclemay include six cameras, seven cameras, ten cameras, twelve cameras,and/or another number of cameras. The cameras may support, as an exampleand without limitation, Gigabit Multimedia Serial Link (GMSL) and/orGigabit Ethernet. Each of the camera(s) is described with more detailherein with respect to FIG. 10A and FIG. 10B.

The vehicle 1000 may further include vibration sensor(s) 1042. Thevibration sensor(s) 1042 may measure vibrations of components of thevehicle, such as the axle(s). For example, changes in vibrations mayindicate a change in road surfaces. In another example, when two or morevibration sensors 1042 are used, the differences between the vibrationsmay be used to determine friction or slippage of the road surface (e.g.,when the difference in vibration is between a power-driven axle and afreely rotating axle).

The vehicle 1000 may include an ADAS system 1038. The ADAS system 1038may include a SoC, in some examples. The ADAS system 1038 may includeautonomous/adaptive/automatic cruise control (ACC), cooperative adaptivecruise control (CACC), forward crash warning (FCW), automatic emergencybraking (AEB), lane departure warnings (LDW), lane keep assist (LKA),blind spot warning (BSW), rear cross-traffic warning (RCTW), collisionwarning systems (CWS), lane centering (LC), and/or other features andfunctionality.

The ACC systems may use RADAR sensor(s) 1060, LIDAR sensor(s) 1064,and/or a camera(s). The ACC systems may include longitudinal ACC and/orlateral ACC. Longitudinal ACC monitors and controls the distance to thevehicle immediately ahead of the vehicle 1000 and automatically adjustthe vehicle speed to maintain a safe distance from vehicles ahead.Lateral ACC performs distance keeping, and advises the vehicle 1000 tochange lanes when necessary. Lateral ACC is related to other ADASapplications such as LCA and CWS.

CACC uses information from other vehicles that may be received via thenetwork interface 1024 and/or the wireless antenna(s) 1026 from othervehicles via a wireless link, or indirectly, over a network connection(e.g., over the Internet). Direct links may be provided by avehicle-to-vehicle (V2V) communication link, while indirect links may beinfrastructure-to-vehicle (I2V) communication link. In general, the V2Vcommunication concept provides information about the immediatelypreceding vehicles (e.g., vehicles immediately ahead of and in the samelane as the vehicle 1000), while the I2V communication concept providesinformation about traffic further ahead. CACC systems may include eitheror both I2V and V2V information sources. Given the information of thevehicles ahead of the vehicle 1000, CACC may be more reliable and it haspotential to improve traffic flow smoothness and reduce congestion onthe road.

FCW systems are designed to alert the driver to a hazard, so that thedriver may take corrective action. FCW systems use a front-facing cameraand/or RADAR sensor(s) 1060, coupled to a dedicated processor, DSP,FPGA, and/or ASIC, that is electrically coupled to driver feedback, suchas a display, speaker, and/or vibrating component. FCW systems mayprovide a warning, such as in the form of a sound, visual warning,vibration and/or a quick brake pulse.

AEB systems detect an impending forward collision with another vehicleor other object, and may automatically apply the brakes if the driverdoes not take corrective action within a specified time or distanceparameter. AEB systems may use front-facing camera(s) and/or RADARsensor(s) 1060, coupled to a dedicated processor, DSP, FPGA, and/orASIC. When the AEB system detects a hazard, it typically first alertsthe driver to take corrective action to avoid the collision and, if thedriver does not take corrective action, the AEB system may automaticallyapply the brakes in an effort to prevent, or at least mitigate, theimpact of the predicted collision. AEB systems, may include techniquessuch as dynamic brake support and/or crash imminent braking.

LDW systems provide visual, audible, and/or tactile warnings, such assteering wheel or seat vibrations, to alert the driver when the vehicle1000 crosses lane markings. A LDW system does not activate when thedriver indicates an intentional lane departure, by activating a turnsignal. LDW systems may use front-side facing cameras, coupled to adedicated processor, DSP, FPGA, and/or ASIC, that is electricallycoupled to driver feedback, such as a display, speaker, and/or vibratingcomponent.

LKA systems are a variation of LDW systems. LKA systems provide steeringinput or braking to correct the vehicle 1000 if the vehicle 1000 startsto exit the lane.

BSW systems detects and warn the driver of vehicles in an automobile'sblind spot. BSW systems may provide a visual, audible, and/or tactilealert to indicate that merging or changing lanes is unsafe. The systemmay provide an additional warning when the driver uses a turn signal.BSW systems may use rear-side facing camera(s) and/or RADAR sensor(s)1060, coupled to a dedicated processor, DSP, FPGA, and/or ASIC, that iselectrically coupled to driver feedback, such as a display, speaker,and/or vibrating component.

RCTW systems may provide visual, audible, and/or tactile notificationwhen an object is detected outside the rear-camera range when thevehicle 1000 is backing up. Some RCTW systems include AEB to ensure thatthe vehicle brakes are applied to avoid a crash. RCTW systems may useone or more rear-facing RADAR sensor(s) 1060, coupled to a dedicatedprocessor, DSP, FPGA, and/or ASIC, that is electrically coupled todriver feedback, such as a display, speaker, and/or vibrating component.

Conventional ADAS systems may be prone to false positive results whichmay be annoying and distracting to a driver, but typically are notcatastrophic, because the ADAS systems alert the driver and allow thedriver to decide whether a safety condition truly exists and actaccordingly. However, in an autonomous vehicle 1000, the vehicle 1000itself must, in the case of conflicting results, decide whether to heedthe result from a primary computer or a secondary computer (e.g., afirst controller 1036 or a second controller 1036). For example, in someembodiments, the ADAS system 1038 may be a backup and/or secondarycomputer for providing perception information to a backup computerrationality module. The backup computer rationality monitor may run aredundant diverse software on hardware components to detect faults inperception and dynamic driving tasks. Outputs from the ADAS system 1038may be provided to a supervisory MCU. If outputs from the primarycomputer and the secondary computer conflict, the supervisory MCU mustdetermine how to reconcile the conflict to ensure safe operation.

In some examples, the primary computer may be configured to provide thesupervisory MCU with a confidence score, indicating the primarycomputer's confidence in the chosen result. If the confidence scoreexceeds a threshold, the supervisory MCU may follow the primarycomputer's direction, regardless of whether the secondary computerprovides a conflicting or inconsistent result. Where the confidencescore does not meet the threshold, and where the primary and secondarycomputer indicate different results (e.g., the conflict), thesupervisory MCU may arbitrate between the computers to determine theappropriate outcome.

The supervisory MCU may be configured to run a neural network(s) that istrained and configured to determine, based on outputs from the primarycomputer and the secondary computer, conditions under which thesecondary computer provides false alarms. Thus, the neural network(s) inthe supervisory MCU may learn when the secondary computer's output maybe trusted, and when it cannot. For example, when the secondary computeris a RADAR-based FCW system, a neural network(s) in the supervisory MCUmay learn when the FCW system is identifying metallic objects that arenot, in fact, hazards, such as a drainage grate or manhole cover thattriggers an alarm. Similarly, when the secondary computer is acamera-based LDW system, a neural network in the supervisory MCU maylearn to override the LDW when bicyclists or pedestrians are present anda lane departure is, in fact, the safest maneuver. In embodiments thatinclude a neural network(s) running on the supervisory MCU, thesupervisory MCU may include at least one of a DLA or GPU suitable forrunning the neural network(s) with associated memory. In preferredembodiments, the supervisory MCU may comprise and/or be included as acomponent of the SoC(s) 1004.

In other examples, ADAS system 1038 may include a secondary computerthat performs ADAS functionality using traditional rules of computervision. As such, the secondary computer may use classic computer visionrules (if-then), and the presence of a neural network(s) in thesupervisory MCU may improve reliability, safety and performance. Forexample, the diverse implementation and intentional non-identity makesthe overall system more fault-tolerant, especially to faults caused bysoftware (or software-hardware interface) functionality. For example, ifthere is a software bug or error in the software running on the primarycomputer, and the non-identical software code running on the secondarycomputer provides the same overall result, the supervisory MCU may havegreater confidence that the overall result is correct, and the bug insoftware or hardware on primary computer is not causing material error.

In some examples, the output of the ADAS system 1038 may be fed into theprimary computer's perception block and/or the primary computer'sdynamic driving task block. For example, if the ADAS system 1038indicates a forward crash warning due to an object immediately ahead,the perception block may use this information when identifying objects.In other examples, the secondary computer may have its own neuralnetwork which is trained and thus reduces the risk of false positives,as described herein.

The vehicle 1000 may further include the infotainment SoC 1030 (e.g., anin-vehicle infotainment system (IVI)). Although illustrated anddescribed as a SoC, the infotainment system may not be a SoC, and mayinclude two or more discrete components. The infotainment SoC 1030 mayinclude a combination of hardware and software that may be used toprovide audio (e.g., music, a personal digital assistant, navigationalinstructions, news, radio, etc.), video (e.g., TV, movies, streaming,etc.), phone (e.g., hands-free calling), network connectivity (e.g.,LTE, Wi-Fi, etc.), and/or information services (e.g., navigationsystems, rear-parking assistance, a radio data system, vehicle relatedinformation such as fuel level, total distance covered, brake fuellevel, oil level, door open/close, air filter information, etc.) to thevehicle 1000. For example, the infotainment SoC 1030 may radios, diskplayers, navigation systems, video players, USB and Bluetoothconnectivity, carputers, in-car entertainment, Wi-Fi, steering wheelaudio controls, hands free voice control, a heads-up display (HUD), anHMI display 1034, a telematics device, a control panel (e.g., forcontrolling and/or interacting with various components, features, and/orsystems), and/or other components. The infotainment SoC 1030 may furtherbe used to provide information (e.g., visual and/or audible) to auser(s) of the vehicle, such as information from the ADAS system 1038,autonomous driving information such as planned vehicle maneuvers,trajectories, surrounding environment information (e.g., intersectioninformation, vehicle information, road information, etc.), and/or otherinformation.

The infotainment SoC 1030 may include GPU functionality. Theinfotainment SoC 1030 may communicate over the bus 1002 (e.g., CAN bus,Ethernet, etc.) with other devices, systems, and/or components of thevehicle 1000. In some examples, the infotainment SoC 1030 may be coupledto a supervisory MCU such that the GPU of the infotainment system mayperform some self-driving functions in the event that the primarycontroller(s) 1036 (e.g., the primary and/or backup computers of thevehicle 1000) fail. In such an example, the infotainment SoC 1030 mayput the vehicle 1000 into a chauffeur to safe stop mode, as describedherein.

The vehicle 1000 may further include an instrument cluster 1032 (e.g., adigital dash, an electronic instrument cluster, a digital instrumentpanel, etc.). The instrument cluster 1032 may include a controllerand/or supercomputer (e.g., a discrete controller or supercomputer). Theinstrument cluster 1032 may include a set of instrumentation such as aspeedometer, fuel level, oil pressure, tachometer, odometer, turnindicators, gearshift position indicator, seat belt warning light(s),parking-brake warning light(s), engine-malfunction light(s), airbag(SRS) system information, lighting controls, safety system controls,navigation information, etc. In some examples, information may bedisplayed and/or shared among the infotainment SoC 1030 and theinstrument cluster 1032. In other words, the instrument cluster 1032 maybe included as part of the infotainment SoC 1030, or vice versa.

FIG. 10D is a system diagram for communication between cloud-basedserver(s) and the example autonomous vehicle 1000 of FIG. 10A, inaccordance with some embodiments of the present disclosure. The system1076 may include server(s) 1078, network(s) 1090, and vehicles,including the vehicle 1000. The server(s) 1078 may include a pluralityof GPUs 1084(A)-1084(H) (collectively referred to herein as GPUs 1084),PCIe switches 1082(A)-1082(H) (collectively referred to herein as PCIeswitches 1082), and/or CPUs 1080(A)-1080(B) (collectively referred toherein as CPUs 1080). The GPUs 1084, the CPUs 1080, and the PCIeswitches may be interconnected with high-speed interconnects such as,for example and without limitation, NVLink interfaces 1088 developed byNVIDIA and/or PCIe connections 1086. In some examples, the GPUs 1084 areconnected via NVLink and/or NVSwitch SoC and the GPUs 1084 and the PCIeswitches 1082 are connected via PCIe interconnects. Although eight GPUs1084, two CPUs 1080, and two PCIe switches are illustrated, this is notintended to be limiting. Depending on the embodiment, each of theserver(s) 1078 may include any number of GPUs 1084, CPUs 1080, and/orPCIe switches. For example, the server(s) 1078 may each include eight,sixteen, thirty-two, and/or more GPUs 1084.

The server(s) 1078 may receive, over the network(s) 1090 and from thevehicles, image data representative of images showing unexpected orchanged road conditions, such as recently commenced road-work. Theserver(s) 1078 may transmit, over the network(s) 1090 and to thevehicles, neural networks 1092, updated neural networks 1092, and/or mapinformation 1094, including information regarding traffic and roadconditions. The updates to the map information 1094 may include updatesfor the HD map 1022, such as information regarding construction sites,potholes, detours, flooding, and/or other obstructions. In someexamples, the neural networks 1092, the updated neural networks 1092,and/or the map information 1094 may have resulted from new trainingand/or experiences represented in data received from any number ofvehicles in the environment, and/or based on training performed at adatacenter (e.g., using the server(s) 1078 and/or other servers).

The server(s) 1078 may be used to train machine learning models (e.g.,neural networks) based on training data. The training data may begenerated by the vehicles, and/or may be generated in a simulation(e.g., using a game engine). In some examples, the training data istagged (e.g., where the neural network benefits from supervisedlearning) and/or undergoes other pre-processing, while in other examplesthe training data is not tagged and/or pre-processed (e.g., where theneural network does not require supervised learning). Once the machinelearning models are trained, the machine learning models may be used bythe vehicles (e.g., transmitted to the vehicles over the network(s)1090, and/or the machine learning models may be used by the server(s)1078 to remotely monitor the vehicles.

In some examples, the server(s) 1078 may receive data from the vehiclesand apply the data to up-to-date real-time neural networks for real-timeintelligent inferencing. The server(s) 1078 may include deep-learningsupercomputers and/or dedicated AI computers powered by GPU(s) 1084,such as a DGX and DGX Station machines developed by NVIDIA. However, insome examples, the server(s) 1078 may include deep learninginfrastructure that use only CPU-powered datacenters.

The deep-learning infrastructure of the server(s) 1078 may be capable offast, real-time inferencing, and may use that capability to evaluate andverify the health of the processors, software, and/or associatedhardware in the vehicle 1000. For example, the deep-learninginfrastructure may receive periodic updates from the vehicle 1000, suchas a sequence of images and/or objects that the vehicle 1000 has locatedin that sequence of images (e.g., via computer vision and/or othermachine learning object classification techniques). The deep-learninginfrastructure may run its own neural network to identify the objectsand compare them with the objects identified by the vehicle 1000 and, ifthe results do not match and the infrastructure concludes that the AI inthe vehicle 1000 is malfunctioning, the server(s) 1078 may transmit asignal to the vehicle 1000 instructing a fail-safe computer of thevehicle 1000 to assume control, notify the passengers, and complete asafe parking maneuver.

For inferencing, the server(s) 1078 may include the GPU(s) 1084 and oneor more programmable inference accelerators (e.g., NVIDIA's TensorRT).The combination of GPU-powered servers and inference acceleration maymake real-time responsiveness possible. In other examples, such as whereperformance is less critical, servers powered by CPUs, FPGAs, and otherprocessors may be used for inferencing.

Example Computing Device

FIG. 11 is a block diagram of an example computing device 1100 suitablefor use in implementing some embodiments of the present disclosure. Forexample, the machine learning model(s) 108 may be executed within any ofa plurality of systems, and the systems may include some or all of thecomponents of the computing device 1100. As such, the underlyingsoftware and/or hardware that may be fault tested using the processes100 and 700 may include one or more components—or components similarthereto—of the computing device 1100. The computing device 1100 mayinclude a bus 1102 that directly or indirectly couples the followingdevices: memory 1104, one or more central processing units (CPUs) 1106,one or more graphics processing units (GPUs) 1108, a communicationinterface 1110, input/output (I/O) ports 1112, input/output components1114, a power supply 1116, and one or more presentation components 1118(e.g., display(s)).

Although the various blocks of FIG. 11 are shown as connected via thebus 1102 with lines, this is not intended to be limiting and is forclarity only. For example, in some embodiments, a presentation component1118, such as a display device, may be considered an I/O component 1114(e.g., if the display is a touch screen). As another example, the CPUs1106 and/or GPUs 1108 may include memory (e.g., the memory 1104 may berepresentative of a storage device in addition to the memory of the GPUs1108, the CPUs 1106, and/or other components). In other words, thecomputing device of FIG. 11 is merely illustrative. Distinction is notmade between such categories as “workstation,” “server,” “laptop,”“desktop,” “tablet,” “client device,” “mobile device,” “hand-helddevice,” “game console,” “electronic control unit (ECU),” “virtualreality system,” and/or other device or system types, as all arecontemplated within the scope of the computing device of FIG. 11 .

The bus 1102 may represent one or more busses, such as an address bus, adata bus, a control bus, or a combination thereof. The bus 1102 mayinclude one or more bus types, such as an industry standard architecture(ISA) bus, an extended industry standard architecture (EISA) bus, avideo electronics standards association (VESA) bus, a peripheralcomponent interconnect (PCI) bus, a peripheral component interconnectexpress (PCIe) bus, and/or another type of bus.

The memory 1104 may include any of a variety of computer-readable media.The computer-readable media may be any available media that may beaccessed by the computing device 1100. The computer-readable media mayinclude both volatile and nonvolatile media, and removable andnon-removable media. By way of example, and not limitation, thecomputer-readable media may comprise computer-storage media andcommunication media.

The computer-storage media may include both volatile and nonvolatilemedia and/or removable and non-removable media implemented in any methodor technology for storage of information such as computer-readableinstructions, data structures, program modules, and/or other data types.For example, the memory 1104 may store computer-readable instructions(e.g., that represent a program(s) and/or a program element(s), such asan operating system. Computer-storage media may include, but is notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical disk storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or any other medium which may be used to storethe desired information and which may be accessed by computing device1100. As used herein, computer storage media does not comprise signalsper se.

The communication media may embody computer-readable instructions, datastructures, program modules, and/or other data types in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” mayrefer to a signal that has one or more of its characteristics set orchanged in such a manner as to encode information in the signal. By wayof example, and not limitation, the communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, RF, infrared and other wireless media.Combinations of any of the above should also be included within thescope of computer-readable media.

The CPU(s) 1106 may be configured to execute the computer-readableinstructions to control one or more components of the computing device1100 to perform one or more of the methods and/or processes describedherein. The CPU(s) 1106 may each include one or more cores (e.g., one,two, four, eight, twenty-eight, seventy-two, etc.) that are capable ofhandling a multitude of software threads simultaneously. The CPU(s) 1106may include any type of processor, and may include different types ofprocessors depending on the type of computing device 1100 implemented(e.g., processors with fewer cores for mobile devices and processorswith more cores for servers). For example, depending on the type ofcomputing device 1100, the processor may be an ARM processor implementedusing Reduced Instruction Set Computing (RISC) or an x86 processorimplemented using Complex Instruction Set Computing (CISC). Thecomputing device 1100 may include one or more CPUs 1106 in addition toone or more microprocessors or supplementary co-processors, such as mathco-processors.

The GPU(s) 1108 may be used by the computing device 1100 to rendergraphics (e.g., 3D graphics). The GPU(s) 1108 may include hundreds orthousands of cores that are capable of handling hundreds or thousands ofsoftware threads simultaneously. The GPU(s) 1108 may generate pixel datafor output images in response to rendering commands (e.g., renderingcommands from the CPU(s) 1106 received via a host interface). The GPU(s)1108 may include graphics memory, such as display memory, for storingpixel data. The display memory may be included as part of the memory1104. The GPU(s) 708 may include two or more GPUs operating in parallel(e.g., via a link). When combined together, each GPU 1108 may generatepixel data for different portions of an output image or for differentoutput images (e.g., a first GPU for a first image and a second GPU fora second image). Each GPU may include its own memory, or may sharememory with other GPUs.

In examples where the computing device 1100 does not include the GPU(s)1108, the CPU(s) 1106 may be used to render graphics.

The communication interface 1110 may include one or more receivers,transmitters, and/or transceivers that enable the computing device 700to communicate with other computing devices via an electroniccommunication network, included wired and/or wireless communications.The communication interface 1110 may include components andfunctionality to enable communication over any of a number of differentnetworks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth,Bluetooth LE, ZigBee, etc.), wired networks (e.g., communicating overEthernet), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.),and/or the Internet.

The I/O ports 1112 may enable the computing device 1100 to be logicallycoupled to other devices including the I/O components 1114, thepresentation component(s) 1118, and/or other components, some of whichmay be built in to (e.g., integrated in) the computing device 1100.Illustrative I/O components 1114 include a microphone, mouse, keyboard,joystick, game pad, game controller, satellite dish, scanner, printer,wireless device, etc. The I/O components 1114 may provide a natural userinterface (NUI) that processes air gestures, voice, or otherphysiological inputs generated by a user. In some instances, inputs maybe transmitted to an appropriate network element for further processing.An NUI may implement any combination of speech recognition, stylusrecognition, facial recognition, biometric recognition, gesturerecognition both on screen and adjacent to the screen, air gestures,head and eye tracking, and touch recognition (as described in moredetail below) associated with a display of the computing device 1100.The computing device 1100 may be include depth cameras, such asstereoscopic camera systems, infrared camera systems, RGB camerasystems, touchscreen technology, and combinations of these, for gesturedetection and recognition. Additionally, the computing device 1100 mayinclude accelerometers or gyroscopes (e.g., as part of an inertiameasurement unit (IMU)) that enable detection of motion. In someexamples, the output of the accelerometers or gyroscopes may be used bythe computing device 1100 to render immersive augmented reality orvirtual reality.

The power supply 1116 may include a hard-wired power supply, a batterypower supply, or a combination thereof. The power supply 1116 mayprovide power to the computing device 1100 to enable the components ofthe computing device 1100 to operate.

The presentation component(s) 1118 may include a display (e.g., amonitor, a touch screen, a television screen, a heads-up-display (HUD),other display types, or a combination thereof), speakers, and/or otherpresentation components. The presentation component(s) 1118 may receivedata from other components (e.g., the GPU(s) 1108, the CPU(s) 1106,etc.), and output the data (e.g., as an image, video, sound, etc.).

The disclosure may be described in the general context of computer codeor machine-useable instructions, including computer-executableinstructions such as program modules, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program modules including routines, programs,objects, components, data structures, etc., refer to code that performparticular tasks or implement particular abstract data types. Thedisclosure may be practiced in a variety of system configurations,including hand-held devices, consumer electronics, general-purposecomputers, more specialty computing devices, etc. The disclosure mayalso be practiced in distributed computing environments where tasks areperformed by remote-processing devices that are linked through acommunications network.

As used herein, a recitation of “and/or” with respect to two or moreelements should be interpreted to mean only one element, or acombination of elements. For example, “element A, element B, and/orelement C” may include only element A, only element B, only element C,element A and element B, element A and element C, element B and elementC, or elements A, B, and C. In addition, “at least one of element A orelement B” may include at least one of element A, at least one ofelement B, or at least one of element A and at least one of element B.Further, “at least one of element A and element B” may include at leastone of element A, at least one of element B, or at least one of elementA and at least one of element B.

The subject matter of the present disclosure is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of thisdisclosure. Rather, the inventors have contemplated that the claimedsubject matter might also be embodied in other ways, to includedifferent steps or combinations of steps similar to the ones describedin this document, in conjunction with other present or futuretechnologies. Moreover, although the terms “step” and/or “block” may beused herein to connote different elements of methods employed, the termsshould not be interpreted as implying any particular order among orbetween various steps herein disclosed unless and except when the orderof individual steps is explicitly described.

What is claimed is:
 1. A method comprising: generating first datarepresentative of sensor data captured by a sensor and second datarepresentative of one or more motifs appended to the sensor data;applying the first data and the second data to a neural network;computing, using the neural network, third data representative ofpredictions corresponding to one or more features represented by thefirst data and the second data; comparing the third data to fourth data,the fourth data representative of expected predictions corresponding tothe one or more motifs represented by the second data; and determiningthat the predictions are accurate when the expected predictionscorresponding to the one or more motifs are represented in thepredictions of the neural network.
 2. The method of claim 1, wherein theneural network is trained for object detection, the one or more featurescorrespond to one or more objects, the expected predictions correspondto actual locations of the one or more motifs, the predictionscorrespond to predicted locations of the one or more objects representedby the second data, and for each motif of the one or more motifs, thecomparing includes determining whether a predicted location of thepredicted locations corresponds to an actual location of the motif. 3.The method of claim 2, wherein the predictions further correspond toclass labels associated with each of the one or more objects.
 4. Themethod of claim 3, wherein the determining that the predictions of theneural network are accurate further includes determining whether theclass labels predicted correspond to motif class labels corresponding toeach motif of the one or more motifs.
 5. The method of claim 1, wherein,when the expected predictions corresponding to the one or more motifsare not represented in the predictions of the neural network, hardwarecorresponding to processing of the neural network is determined to havea transient fault.
 6. The method of claim 1, wherein, when the expectedpredictions corresponding to the one or more motifs are represented inthe predictions of the neural network, the predictions of the neuralnetwork are used to perform one or more operations.
 7. The method ofclaim 6, wherein the sensor is an image sensor of a vehicle, and the oneor more operations are associated with semi-autonomous driving orautonomous driving functionalities of the vehicle.
 8. The method ofclaim 1, wherein, when the expected predictions corresponding to the oneor more motifs are not represented in the predictions of the neuralnetwork, the performance of one or more operations is suspended orterminated.
 9. The method of claim 1, wherein, at each iteration of thefirst data and the second data, at least one of a location, a number, ora motif class of at least one motif of the one or more motifs is changedwith respect to a preceding iteration of the first data and the seconddata.
 10. A method comprising: applying; to a neural network, first datarepresentative of a plurality of sensor data instances captured by atleast one sensor; applying, intermittently during the applying the firstdata to the neural network, second data representative of a signatureinput to the neural network; comparing third data representative of anoutput of the neural network computed based at least in part on thesecond data to fourth data representative of an expected output of theneural network based at least in part on the second data; anddetermining that predictions of the neural network are accurate when theoutput corresponds to the expected output.
 11. The method of claim 10,wherein hardware corresponding to processing of the neural network isdetermined to have a permanent fault when the output does not correspondto the expected output.
 12. The method of claim 10, wherein predictionsof the neural network are used to perform one or more operations whenthe output corresponds to the expected output.
 13. The method of claim10, wherein the applying the second data intermittently includesapplying the second data at one of an interval of time or every x numberof the plurality of sensor data instances.
 14. The method of claim 10,wherein the neural network is used for at least one of semi-autonomouscapabilities or autonomous driving capabilities of a vehicle; and theapplying the second data corresponds to a built-in self-test (BIST)system of the vehicle during operation.
 15. The method of claim 10,wherein the signature corresponds to an image including at least one of:one or more motifs, a stock image, or a logo.
 16. The method of claim10, wherein the output corresponding to the expected output includesfirst values represented by the output being within a threshold distanceto second values represented by the expected output.
 17. A systemcomprising: one or more sensors to generate first data; a computingdevice including one or more processing devices and one or more memorydevices communicatively coupled to the one or more processing devicesstoring programmed instructions thereon, which when executed by theprocessor causes the instantiation of: a motif appender to generatesecond data by appending one or more motifs to at least one iteration ofthe first data; a predictor to compute, using a neural network and basedat least in part on the second data corresponding to the at least oneiteration of the first data, third data representative of predictions ofthe neural network corresponding to one or more features represented bythe first data and the second data; and a fault detector to determinewhether a fault is present based at least n part on comparing the thirddata to fourth data representative of expected predictions correspondingto the one or more motifs.
 18. The system of claim 17, wherein thepredictions of the neural network correspond to a predicted locations ofobjects in the second data, and the comparing the third data to thefourth data includes comparing the predicted locations of the objects toactual locations of the one or more motifs.
 19. The system of claim 18,wherein the predictions of the neural network further correspond to aclass labels of objects in the second data, and the comparing the thirddata to the fourth data includes comparing the predicted class labels ofthe objects to actual class labels of the one or more motifs.
 20. Thesystem of claim 17, further comprising: a control component to performone or more operations based at least in part on the determining whetherthe fault is present, the one or more operations being differe when thefault is detected than when no fault is detected.
 21. The system ofclaim 17, wherein the one or more motifs are appended to the first dataat a first iteration of the first data and a second iteration of thefirst data, and the one or more motifs differ between the firstiteration and the second iteration with resp at least one of location,number, type, or class.